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ABSTBACT • v * 

" ^ This report discusses the development of procedures 
for establishing teacher estimates of minimal acceptable, desired , 
and predicted levels of student performance. on specific reading items 
and across reading skill areas. Statewide estimates were obtained by 
collecting and averaging item estimates from saiples of teachers 
throughout the state, and district-vide estimates were obtained by 
collect incite* estimates from groups! of teachers using consensus 
procedures. Item estimates have been compared directly to student 
performance on the items, and individual item estimates have been 
"averaged across reading skill areas ilor comparison with student 
performance across s fch^same skill ar4as* The minimal acceptable and 
desired estimates across skill areas ijiave been used as criterion 
measures to establish relative strengths and weaknesses in student 
performance. These criterion measures? have also been used in 
conjunction vi&h "normative measures across the same skill areas to 
establish a more complete profile of %he adequacy of student 
performance. Teacher outcomes studies have been conducted in three 
assessment projects. The pilot study va^s conducted during the t 1 973-74 
9-year-old Minnesota Reading Assessment \ The questionnaire contained 
22 items across four reading domains ano^ various difficulty levels. 
The 22 items were selected fro* the 189 items administered to 
students in one, of three .independent stiident samples. The teacher 
samples consisted of third and fourth grade classroom teachers and 
reading specialists in a subsample of the schools in each of the 
student samples. (Author /DEP) 
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One area of concern in testing and assessment has been that of estab- 
lishing a ^standard of performance, or a criterion, against which student 
performance can be measured in making decisions about changes in curriculum 
and instruction* This paper discusses the development of procedures for 
establishing teacher estimates of minimal Acceptable, desired, and predicted 
levels of student performance an specif f c . reading items and across reading 
skill areas. Statewide estimates were obtained by collecting and averaging 
item estimates from samples of teachers throughout the state, and district- 
wide estimates were obtained by Collecting item estimates from groups of 
teachers u^ing consensus procedures* Item estimates have been compared 
^directly to student performance on the items and individual item estimates 
have been averaged across reading skill areas for comparison with student 
performance across the same skill areas ^ The minimal acceptable and desired 
estimates across skill areas have been used as criterion measures to estab- 
lish relative strengths and weaknesses in student performance^. These cri- 
terion measures have also been used in qon junction with normative measures 
acfoss* the same skill areas to festablisti a more complete prQf ile of the 
adequacy of student performance. f ^ 

Teacher outcomes studies have been-* conducted in 3 assessment projects* 
The pilot' study was conducted during the 1973-1974* 9-year-old Minnesota § 
Reading Assessment* Designed by Research Triangle Institute, Minnesota 
Department of Education, and University of Minnesota Department of Educa- 
tion staff, the questionnaire contained 22 items across 4 reading domains 
and various difficulty levels* The 22 items were selected from the 189 
items administered to students in 1 of 3 independent student samples* The 
teacher samples consisted of third and fourth grade classroom teachers and 
reading specialists in a subsample of the| schools in each of the student 
samples. Each booklet contained the samei22 items with instructions to 



make 3 estimates (in intervals of 10%) of 



mal acceptable outcome, the desired outcome, and the predicted outcome 
These were defined— as follows: 



student perf ormance--the mini - 



1. Minimal Acceptable Outcome - The percent of Minnesota 9-year- 
olds you Relieve must be able to xjespond correctly to a particu- 
lar reading N item in order for you jto consider reading instruction 
to be providing essential reading 
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— ^More complete definitions of these 
ment la. 

t 

7T"-" i 3 



skills to these students* 
terms may be found in attach- 



2* desired Outcome - The percent of Minnesota 9-year-olds you believe 1 

should he able to respond .correctly to a particular reading item. 
3. Predicted Outcome - The percent of Minnesota 9-year-olds you 
believe will .respond correctly to a particular reading item. 
The booklets were distributed to' the school principals at the time of 
the student testing. The principals distributed the booklets to their 
teachers; and each teacher was requested to complete the booklet and return 
it to the Minnesota Department of Education within f 10 days. Each .'packet 
included a letter from the Commissioner of Education requesting the teachers 
cooperation, a fact sheet about the assessment program, and a self-addressed 
envelope for returning the questionnaire. - A few weeks after the distribu- 
tion, a letter was sent to- each/principal indicating the number of teachers 
in the school who had not returned their booklets and requesting cooperar 
"tion in obtaining these. The /booklets were collected at the Minnesota 
department of Education and forwarded to the\Research Triangle Institute ^ 
for coding and analysis. 

Of the 762 teachers selected, 620* responded.- Mean estimates for mini 
mal acceptable, desired, and predicted outcomes were calculated for each 
'item by sample, and overall mean estimates and standard errors were calcula- 
ted across .the 3 samples. These are shown in attachment lb. The 3 types 
of outcomes were also calculated by type of teacher: 

a. Third grade classroom teachers 

b. FouV^h grade classroom teachers 

c. Reading specialists - third and fourth grades only 

d. Reading specialists - all elementary grades, 
itimates are given in attachment Ic. 

This pilot study demonstrated the technical feasibility and potential 
educational utility of the teacher outcomes approach. The teachers' esti- 
mates for each item were relatively stable within and across the 3 samples, 
indicating basic reliability of the instrument. Table 1 shows that the 
teachers' estimates of predicted performance were generally "close to actual 
student performance. For 17 of the 22 items, the teachers' estimates 

^The 3 samples were independent, consisting df all eligible teachers 
in a subsample of the schools in each of the 3 student samples. Schools 
could be (and occasionally were) selected- for more than 1 sample. Thus* 
.some teachers were in more than 1 sample. The numbers of teachers respond- 
ing in samples 1, 2, and 3 were 232, 237, and 251, respectively. 
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. 4 Table 1 
Minnesota Teacher Outcomes Study 



Teacher Estimates 



Item 



Minimal 

Acceptable Desired Predicted 



Actual 

Student 

Performance 



Teachers' 
Prediction- 
Student 
Performance 



1A 


43.4 


,73.3 


59,-5 


50.3 


9.2 


IB 


35.6 


. 64.4 


47.6 , 


57.8 


-10.2 « 


2A 


56.2 


• 84.5 


71.2 


79.7 


-8.5 


2B. 


59.3 


* 86.8 


74.8 


90.4 


-15.6 


3A « 


47.6 


. 76.1 


60.0 




-13.9" 


3B 


46.5 


,75.1 


58.7 % 


52.8 


• 5.9 


4A 


57.3 


86.3 


73.4 


87.1 


-13.7 


4B 


46.7 


75.8 


59.7 


74.1 


-14.4 


5A 


36.7 


65.7 


47.4 


37.0 


10.4 


5B 


42.4 


/ 70.7 


55.6 


58,4 


-2.8 


6A 


41.3 


70.1 A 


53.9 


60.8 


-6.9 


6B 


46-. 5 


74.9 


60.6 


78.9 


-18.3 


7A 


44.8 


74.9 


59.5 


.76.2 


-16.7 


7B 


50.2 


79.5 


66.4 


71.0 


-4.6 


8A 


46.4 


76.4 


61.2 • 


73.2 


-12.0 


8B 


52.7 


\ 82.2 


68.9 


76.4 ' 


-7.5 


9 


41.3 


71.2 


52.7 • 


33.6 • 


19.1 


.10A 


62.4 


90.6 


79.2 


87.8 


-8.6 


10B 


54.4 


83.9 


69.1 


90.4 


-21.3 


-11 


50.7 


80.2 


64.9 


77.4 


-12.5 


12 


47.9 


77.8 


62.4 


68;6 


-6.2 


13 


46.9 


77.6 


60.9 


59.9 


1.0 



of predicted outcomes fell within 15 percentage points of the actual per- 
formance; for 10, they fell within 10 percentage points of the actual; and . 
for 3 they fell within" 5 percentage points of the actual student, per- 
formance. For 5 items, the student performance was above the "teachers- 
mean desired level T and for 1 itemi tne 'student, performance was below the 
minimal acceptable level'. The level of accuracy In predicting student per- 
formance indicates that__the teachers have a reasonable perception of the 
capabilities of Their students and lends confidence in using their subjec- 
tive judgments of minimal and desired outcomes in determining the accepta- , 
bility of student reading performance. 

A survey of a sample of the responding teachers provided further evi- 
dence that the instrument was comprehensive and potentially useful. A 
questionnaire was sent to 30 of the responding teachers to determine their , 
understanding of the instructions and format of the original booklet and . 
their attitudes toward its potential usefulness. The responses were .generally 
favorable. Almost all of the teachers said they understood the -definitions , . 
_thojxgh-most of them said they felt uneasy estimating statewide performance. 
These teachers felt the study could be useful. The results of this survey 
may be found in attachment Id. 

This pilot study was followed by 2 other studies "designed to increase 
* the usefulness, of the teacher- outcome's process in establishing criterion 
levels for student performance. In Maine a statewide instrument was used 
in which teacher, estimates were obtained for a more adequate sample of 
reading items measuring certain reading skills. In the Richfield, Minnesota, 
Public School District, groups of teachers used consensus procedures to 
establish criterion levels for all reading" items in the assessment. ^ 

In the 1973-1974 9-year-old Reading and Writing Assessment in Maine, 
"the teacher outcomes booklet consisted of 15 jading items from 2 themes, 
with complete coverage of all the items measuring 1 theme. The scoring 
information for open-ended items was included. As in the Minnesota study, 
the teachers were requested to make statewide estimates of minimal accepta- 
ble, desired, and predicted outcomes. Three samples of teachers were 
selected -by subsampling the schools in the 3 student samples. Data 
collection was conducted as in the Minnesota study. 
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Table 2 



Maine Teacher Outcomes Study-Item Results 



Teacher Estimates - 



Actual 
Student 



Teachers 1 \ 
Prediction- 
Student 



Minimal 

Item' Acceptable Desired Predicted Performance Performance 



1 




51.3 


80.8 


00. O 


/ J. V 


-6 2. 


2 




55.1 . 


82.4 


^69.5 


ft! S 
OX . J 


-12.0 


3 




34.4 


61.0 


4A.4 


32.2 


\ 12.2 


4 




44.9 


72.5 


56.4 




16 1 


5 * 




54.4- 


on f 

81.6 


DO. 3 


RA ft 
o*i . o 


-16.3 


U 




50.5 


76.5 


62.4 ¥ 


59.9 


2.5 


7A 


■ 


57.2 


85.0 


72.0 


75.3 


-3.3 


7B 




53.9 


79.5 


66.2 


52.0 


14.2 


8 




45.6 


, 72.7 


57.2^ 


68*8 


-11.6 


9A 




50.1 


78.1 


63.6 


68.3 


-4.7 


9B 




43.3 


.71.3 


55.8 


62.1 


-6.3 


10 
11 




55.0 
54.0 


82.6 
81,3 


^^68.2 
*" 67.4 


74.3 
86.4 


-6.1 
-19.0 


12 


V 


57.3 


83.9 


T^.l 


67.8 


4.3 


13 




40.1 


69.1 


51.5 


35.7 


15.8 




Of 


the 281 


teachers 


\ 

selected in the 


samples , 


227 responded. 



y The 

mean responses and corresponding standard errors for the 3 estimates for 
each item were calculated within each sample and these were pooled for the 
overall estimates (see attachment lib). Due to the relatively small sample 
sizes in the Maine study, separate estimates were not made, by type of 
teacher*. 

Table 2 shows that, a* in the Minnesota study, the teachers' predictions 
came relatively close, to the actual student performance. For 11 of the 15 
items, the teachers'" predicted outcomes fell within 15 percentage points of. 
the actual performance; for seven, within 10 percenatges po/nts; and for 4 
within 5 percentage points of the actual student performance. In 4 cases 

lUs in the Minnesota study, the samples were independent and there 
was some overlapping. After deletion of the nonrespondents , samples one, 
2',' and 3, contained 61, 90, and 90 teachers, Respectively. 




student performance fell below the teachers 1 minimal acceptable level ' 

of performance; and in 2 cases performance was above the teachers 1 desired 

level. N < 

. Nine of %he items measured -"reading for main idea and organization" 
(Theme. 6) and 6 items measured "r.eading and drawing inferences" (Theme 7). 
Mean performance estimates were calculated for the sets of items in order 
to obtain criteria against which to judge overall student performance oh 
these skills, As may be seen in table 3, the student performance for these' 
' groups of items and on the total of 15 items was dlose to the teachers 1 pre T 
dieted level of performance ♦ 

As in the Minnesota study, the teacher outcomes study in Maine was 
followed by an opinion survey (attachment lid). A questionnaire similar .to 
that used in Minnesota was sent to 25 teachers who had responded to the 
original questionnaire. The teachers reported that they understood the 
definitions though they felt uneasy estimating statewide performance. 
Generally, the teachers felt that the study was potentially useful. 

The third teacher outcomes study was conducted in the Richfield Public 
Schools in Richfield, Minnesota. Richfield administered *dl Minnesota 
Reading Assessment items to samples of students at ages 9 and 13. Commit- 
tees of teachers at each age level estimated minimal acceptable, desired, 
and predicted outcomes for all assessment items using consensus procedures. 

V The 9-year-old committee consisted of 3 third and A fourth grade 
teachers, 1 teacher from each of >^he 7 elementary buildings in the dis r 



Table 3 

Maine Teacher Outcomes Study — Theme Results 



teacher Estimates 



Actual 
Student 



Teachers 1 
Prediction- 
Student 



Minimal 

Acceptable Desired Predicted Performance Performance 



Theme 6 (9 items) 49.5 
/Theme 7 (6 items) 50.0 
\ Total (lS^tems) 49.7 



76.5 
77.7 
77.0 



r 



63.1 
65.8 
64.2 



-0.7 
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trict. The 13-year-old committee consisted of 7 English teachers,. 4 from 
1 junior high school and} .3 from the other.' The committee members were 
giver. 2 days of released time.Irom their teaching responsibilities te per- 
form their functions" as the "Teacher Outcomes Committees.". 

•* The committees met tegether for half a day for training by Research ( < 
Triangle Institute, University of Minnesota, and Richfield School District 
staff. They were given? the same definitions of minimal acceptable, desired, 
and predicted outcomes as were used in the statewide Minnesota and Maine 
studies. In addition, the teachers were given°all assessment items, nec- 
essary background materials, and special booklets for recording their esti- 
mates for Richfield students. After the training session, ^the 2 groups 
began their work; each. group appointed a chairperson to keep\the work on 
schedule during the day and .a half allotted f or-completion of the task and 
to list the item estimates^ a master booklet. A member of the Richrteld^ 
central staff was available to answer questions about definitions and 7 pro- 
cess. After completion, copies of the master booklets were sent to f the 
Research Triangle Institute for analysis. 

The committee' members represented a wide range of classroom situations- 
from remedial to advanced ability groups; and , in using the consensus pro- 
cess, they had an opportunity t to interact about student performance. As 
shown in table 4, more. extreme estimates were obtained in the Richfield 
consensus study on items identical to those in the statewide Minnesota 
study. The Richfield predicted outcomes for these items ranged from 20 to 
98, whereas the statewide estimates ranged only from 47.4 to 74.8'. The \ 
differences in ranges are similar for minimal acceptable and desired out- 
comes. ' • i » 

Attachment Ilia presents the minimal acceptable, desired, and predicted 
outcomes with the actual Richfield student performance for. all items. Of 
the 189 items at the 9-year-old level, the teachers' predicted level on 131 
items was within 15 percentage points of the actual student performance; on 
95 items performance was within 10 percentage points; and on 58 items it 
was within 5 percentage points of the student performance level. In 31 
cases student performance was below the teachers' minimal acceptable level; 
and in 63 cases, it was above the mean desired level. For 138 items out of 
194 at the 13-year-old- level, the teachers' predicted level was within. 15 
I 
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- ^ Table 4 

^Richfield Teacher Outcomes^ Study 
Items Overlapping the Minnesota Teacher Outcomes Study 



Minimal Acceptable 
Outcome 



Desired ' \ 
Outcome 



Predicted 
Outcome 



Item State Richfield State Richfield 



State Richfield. 
^ 



1A 

IB 

2A 

2B 

3A, 

3B 

• 

AA 
4B 
5A 
5B 
6A 
6B 
7A 
7B 
8A 
8B 

<r 

10A 

10B 

11 

12 

13 



43,4 
35.6 
56.2 
59.3 
47.6 
46.5 
57.3 

. 46.7 
36.7 
42.4 
41.3 
46.5 
44.8 
50.2 
46.4 

12.7 
41.3. 
62.4 
54.4 
50.7 
47.9 
46.9 



55 
30 
50 
75 
45 
45 
85 
60 
15 
25 
65 
40 
50 
60 
40 
65 
50 
95 
80 
70 
50 
80 



73.3, 

64.4 

84.5 

8C.8 

7<>.l 

75.1 

86; 3 

75.8 

65.7 

70.7 

70.1 

74.9 

74.9 

79.5 

76.4 

82.2 

71.Z 

90.6 

83.9 

80.2 

77.8 

77*6 



75 

1 

60| 
70 

1 

90 
75 
75 
95 
75 
30 
55 
90 
80 
75 
80 
80 
90 S 
75 
100 
90 
90 
75 
90 



59.5 

47.6 

71.2 

74.8. 

60.0 

58.7 

73.4 

59.7 

47.4 

55.6 

53.9. 

60.6 

59.5 

66.4 

61.2 

68.9 

52.7 

79.2 

69.1 

64.9 

62.4 

60.9 



70 
45 
55 
80 
60 
60 
90 
70 
20 
35 
75 
55 
65 
70 
60 
80 
65 
98 
85 
80 
60 
85 



r 



10 



percentage points of actual student performance. For 112 items the predicted 
was within 10 percentage points of the actual, and for 70 items the 1 predicted 
level was within 5 percentage points of student performance. Performance 
was below the "teachers' minimal acceptable level on 35 items and, above the 
desired level on 18 items. 

The Richfield study included all items at the 9- and 13-year-old levels 
so that item estimates could be clustered by reading skill. The items 
measured 4 domains, 11 objectives, and ; 24 subobjectives at the 9-year-old 
level and 4 domains, 13_objectives, and 29 subobjectives at the 13-year- 
old level: (ieai) student performances and standard errors and mean teacher 
estimates of minimal acceptable, ddsired, and predicted outcomes for the 
items representing each domain, objective, and subobjective were calculated. 
As may be seen in attachment Illb, tfrom a total of 36 domains, objectives, 
and subobjectives ^at the 9-year-old\ level, the ^teachers' mean estimate of- . 
predicted performance on 30 was within 15 percentage points of the actual 
student performance; /the predicted livel on 27 -was within 10 percentage 



points; and the predicted on 13 was within 5 percentage pointk of the actual. 
There were 7 domains, objectives, or ^objectives for which student per- 
formance fell" above' the teachers' desiVed level of performance and 3 for 
which studeSt performance fell below tile teachers' minimal l^jvel. 

\kt the 13-year-old\ level, there: were a total of 43 domains, objectives, 
and subobjectives.' In 36 cases the teachers' mean predicted level fell 
within 15 percentage points of student performance; in 31 cafces the predicted 
leyel was within 10 percentage, pojftts; and in 23 cases the predicted level 
was wfthin 5 percentage points of student performance. There were no 
domains, objectives, or subobjectives for which student performance fell 
above the desired level; but in 6 of the 43, student performance fell 
below the teachers' mean estimate of a minimal acceptable level 

A method was devised for determining the\ strength or weakness of stu- 
dent performance in relation to the teachers' estimates of minimal and 
desired outcomes. The midpoint of the minimal Vto desired range was deter- 
mined and the Richfield confidence interval was\ calculated.- Figure i 
defines the rule for assigning classifications <\f "Need", "Potential Need", 

i/The confidence interval' (a = .05)' is equaSt to the Richfiefci mean per- 
formance plus and minus twice the corresponding standard error. 
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"Potential Strength", and "Strength" to student performance. If the confl- 1 
dence interval of the Richfield mean performance level included the midpoint 
of the Minimal to Desired range, student performance was considered to he 
Jutral, neither a strength nor a weakness. Table 5 presents an example of 
each classification i taken/from the 9-year-old study. Tables 6 and 7 
fgtijpw that ^his, method allowed categorization of student performance • across 
^Pi^y entire range of strengths and needs. J 
^;j0 The indicationTof strengths and weaknesses within /he various reading 
ggikill areas will be analyzed* along with otW. data,- by reading educators 
and Richfield teachers to determine changes which should be made in the , •• 
reading program.- The process of using classroom teachers to make the esti- 
mates focuses attention on educational objectives and student capabilities. 
The use of estimates established by instructional staff provides locally 
developed, meaningful criterion levels against which student performance / 
can be compared. • \ i 
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Table 5 
Examples of Classifications 
9-Year-01ds 



IVC3 (3) 
IVA* (11) 
IIA* (20) 
IIB1 (8) 
TA6 (8) 



Performance 
» 

Z SE 



62.5 1.60. 

57.4 1.60 
72.1 1.16 

70.5 1.64 
78.4 1.45 



95ZC.I. 

* * 


Teacher estimates 




Class 


Minimal 


Midpoint 


Desired 


59.3-65.7 


71.7 . 


79.2 


86.7 


N 


54.2-60.6 


55.5' 


66.2 


76.8 


PN 


J&k. 8-74. 4 


63.5 \ 


,71.6 


79.8 




67,2^*3.8 


48;i 




76.3 


PS 


75.5-81^\ 




* 56.0 


68.8 


Si 




11 



13 




Table 6 

Classif ication of Student Performance 

* • 

\ 9-Year-01ds 



Potential Potential 
Need Need Neutral Strength Strength 



Domains (4) 
Objectives (11) 
SubobjectiVes (24) 



0 
0 

3 



0. 
2 
4 



1 

2 

4 



2 
5 
8 



.2 
'5 



Table 7 

Classification of Student Performance 
13-Year-Olds 



Potential Potential ^ 

Need Need , Neutral Strength Strength 



-Domains (4) 
Objectives (13) 
; Subobjectives (29) 



0 

1 
6 



2 / 
1 



1 
3 
3 



1 
5 
13 



0 
0 
0 




\ 

V 



Attachment la -~" 
MINNESOTA TEACHER OUTCOMES STUDY 
Excerpts from the Outcomes Booklet 
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MINNESOTA STATEWIDE EDUCATIONAL ASSESSMENT 



TEACHER OUTCOMES BOOKLET 



Place Label Here 



o 

Reading 
Year 01 
9-Year-01ds 
February - March, 1974 



Conducted By: 
The Minnesota Department of Education 
With The Assistance Of 
The Research Triangle Institute 
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INTRODUCTION 



This booklet contains some of the reading items which will be administered to 
a sample of 9-year-old Minnesota students during this February and March. FoV 
each itaL please indicate three percentages relating to Minnesota 9-year-olds. 
First, IndWte your minimal acceptable outcome or the percent of Minnesota 
9-year-olds you^belieJe must be able to respond correctly, to the item in order 
for you to consider that reading instruction in the state meets the most basic 
needs of these students. Second, indicate your desired outcome or the percent 
of Minnesota 9-year-olds you would like to see respond correctly to the item. 
Finally, please iniicate your, predicted outcome or the percent of Minnesota 
9-year-olds you believe will actually .respond to the item correctly* 



Here are some questions you nay have as you prepare for this task. The answers 
are designed to help you estimate the percentages. 



1. How is a 9-year-old defined for this assessment? 



A 9-year-old is a student born during the calendar year 
1964. Approximately three-fourths of them will be in 
the fourth grade, and approximately one-fourth of them 
will be in the third grade. A few may b6 in the second 
or fifth grades. / 




II I don't feel comfortable estimating state percentages, should I 
my percentages upon my class or students in my school? 

You should attempt to base your percents upon students 
throughout the state of Minnesota, "The degree to which 
you will feel comfortable doing this will depend upon 
your training and experience. In practice you will haVe. 
to relate to your own experienc es wl L h s tud ents. Please . 
generalize to the. state population as much as your expe- 
rience will allow. 



\ 



\ 



The following may help you in determining percents. 
Suppose 30 9-year-olds^represented the total population 
of 9-year-olds in Minnesota. If so, the chart below 
would hold true: 



Number of 
Students 


Percent of 
Students 


30 


100% 


27 


90Z 


24 


80Z 


21 


70Z 


18 


60Z 


15 


50Z 


12 


40Z 


9 


30Z 


6 


20Z 


3 


10Z 



What is meant by minimal acceptable outcome? X 



This is the percent of Minnesota i-year-olds you believe 
mist be able to respond correctly ko a particular reading 
Item in order for you to consider reading instruction to 
be providing essential reading skills to these students. 
To determine the minimal acceptable outcome most people 
will probably, consider the importance of the material being 
measured by the item in terras of mastering future reading 
skills for reading at a level necessary to operate success- 
fully in society. 
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The minimal ac^ptable outcome might be viewed in another 
way. If the pejrcent of Minnesota 9-year-olds responding 
correctly to the item wsre below this percent, you would 
feel very concerned about the instruction of the reading 
skill measure*! by that item. 

What is meant by desired outcome? 



this is the percent of Minnesota 9-year-olds you believe, 
should be able to respond correctly to a particular reading 
item. If reading instruction were optimal for Minnesota \ 
students, this is the percent that Would respond correctly V 
to the particular exercise. To determine desired outcome, \ N 
most people will probably consider' the importance of the 
material being measure by the item as well as the difficulty 
of the item in terms of the general abilities of 9-year-olds. 

The ^desired outcome might be viewed in another way. If the 
percent of Minnesota SHyear-olds-responding correctly to the 
item were above this percent, you would feel rather satisfied 
about the instruction of the reading skill measured by that 
item. / ^ 4^ 

ft ■ ■ 

What is meant by predicted outcome? 



This is the percent or Minnesota 9-year-old? you believe will 
respond correctly to a particular reading item. * To determine 
predicted outcome, most people will consider the level of 
reading instruction in the state, as they perceive it, along 
with the difficulty of the item in terms of both the general 
abilities _of 9-year-olds and the f ability of 9ryear-olds to 
master the particular item. with prcteent reading instruction* 

Can a predicted outcome' ever be higher than a desired outcome? 

' . s \ 

Yes. For instance, you may believe present reading instruction 
emphasizes mastety of certain skills which are of minor j 
importance to good reading instruction. If so, you may 
feel that the actual outcome will be better than your desired 
outcome on the particular item. ^ 
1 - \ 
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DIRECTIONS 



The next few pages-^ntain some of the reading items which will be administered 
to 9-year-olds in Minnesota. Examples used to prepare, the students to respond 
are included with each item. The tape scripts, which show what was read to the 
.students on 'audio .tape as they read the item to themselves, are also Included. 
For eachftem, students were given an amount of time considered adequate for most 
Students to respond without time pressure. ( 

— . , ♦ " . . 

For each item, there is a place to Indicate your estimates of the minimal accept- 
able outcome, desired outcome, and predicted outcome. Please glace an in the 
box representing your estimate of each percent , to the nearest 10% . 

* ' " " \ 



EXAMPLE- 



The boxes you are to complete for each item are shown below. In this example, 
tike estimate for the minimal acceptable outcome is 40%. If the actual outcome 
wire below 40%j the teacher would feel very concerned about instruction of the 
reading skill measured by the particular item. The est-ijaate for the desired 
outcome is 70%. If the actual outcome were above 70%, the teacher would, feel 
rat-her satisfied about instruction of the reading skill measured by the particular 
item. In this example, the estimate for the 1 predicted outcome^is 60%. The 
teacher believes 60% of Minnesota 9-year-olds will actually respond to the partic- 
ular-item correctly. 



\ 1 

percent 


10 


20 


30 
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ITEM 1 . AS PRINTED 

•In this exercise we want to find out how well/you can recognize .the number of 
Enables in a word. For *ch part read the, underlined key word and. decide how 
many syllables it has. Then fill in the oval next to the numoer.oi syllables x 
the Underlined word. ' «. 



Example .1 




Example 2 ° 






The word between has 


• / 


The word medicine has 






CD I syllable. 
C9> 2 syllables. 
CD 3 syllables. 
CD^A syllables. 


. / 

/ 

/ 
/ 


CD 1 syllable. 
CD 2 syllables. 
1 CD 3 syllables. 
CD 4 syllables. 


-* 




CD I don 1 ^ know. 

* / ' 


, 1 

i 

/ 


CD I don't know. 







A. The word strewed .has 

CD 1 syllable. 
CD 2 syllables. 
~^~>CD 3 syllables. 
^3>4 -syW*bl e s . 

CD 1 don't know. / 



/ 



/ 



PeAcznt 


,10 


20 


30 


UO 


50 


60 


70 


80 


90 


100 


UiyujnaZ / 












* 










/ 

VuaMjA/ 
VhzdLtted 











































B. The word inaugural has 

CD 1 syllable. 
CD 2 syllables. 
CD* 3 syllables. 
CD 4 syllables. 



CD I don't know. 



Indicate. ijouA utXmatzt fan 18 hvtti 



PeAt&nt 


10 


20 


30 


no 


50 


60 


70 


00 


90 


100 


Hinimat 
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TAPE SCRIPT FOR ITEM 1 



i 



\ 

f » 

\ 

In this exercise we want to find out how well you can recognize the. number of 
syllables in a word— For each part, read the underlined key word and decide how 

miny syllables it has. Then fill in the oval next to the number of syllables 

( " I 

in the underlined word. Look at Example 1 in^the box. 

(Pause :05) « 
Notice that the oval next to two syllables has been filled in because the word $ 
between has two syllables. Now yoli do Example 2 in the box; Read the underlined 
key word and decide how many syllables it has. Fill in the oval next to the 
number of syllables in the underlined key word'. 

(Pause :10) / 
Did you fill in the oval next to three syllables? Three syllables is the correct 
answer because the word medicine has three syllables. If you did not fill in the 
oval next to three syllables , please do so now. 

(Pause :10) 

Now you do Parts A and B on your 6wn. Remember, for each part fill in the number 
of syllables in the underlined word. Ready? Begin. jI 
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ITEM 2. AS PRINTED 



In this exercise L want to sec how well you can choose a word that has the same 
vowe I sound in the middle as the. sound you hear in the middle of a word I say You 
ST to listen for the middle vowel, sound of. a key word that I say and then fill in 
the oval next to the word that has the same vo>.el sound m the middle 



Example I 


Example 2 ' 


CD sir 
CD tide 

€B9 pi n " * — - - 

I— J tlc 


CD cat 
CD cart 
CD map 

CD rate 

■ 


n/I don ! t know. » 


f-~) I don f t know. 


Notice that the oval next, to "pin" 
has been filled in because the vowel 
in the middle of "pin" has the same 
sound as the vowel in the middle of 
"sit." 


You should have filled in the oval- 
next to "rate" because the vowel in 
the middle of "rate" has the same 
sound as the vowel in the middle of 

it it 
cape. 


f ~ 





(See Tape Script for key word— "lob.") 

' A. CD told 
CD dot 
CD for 
CD toy 



CD ! don't know. 



Indicate, ijoux utimatu fai 2A keAti 



PeAce>it 


10 


20 


30 


40 


50 


60 


70 


' 80 


90 


100 


Peaked 
Vnzdictzd 
























> 





































(See Tape Script for key word — "might.") 

B.. CD tin I 
CD fir I 
O kit 
CD bite 

CD I don ! t know. 



Indicate. youA MtuncuteA fan 28 fiete: 



PoAccnt 


10 


20 1 


30' 


uo 


50 


60 


, 70 


80 


90 


100 


WiinvwaJL 
Pelted 
VfitdioJtzd 
























* 





























TAPE SCRIPT FOR ITEM 2 



In this exercise we- want to see how well you c£n choose a word that has the same 
vowel sound in the middle as the sound you hear in fhe middle of a word I say. 
tfou are to listen for the middle vowel sound of a key word that I say and fill in , 
the oval next to the word that has the same vowel sound in the middle. Look at 
Example 1 in the box as you listen to the key word. The key word is sit (Pause) 

sit- i , • 

(Pause :05) ( 
Notice that the oval next to pin has been filled in because the vowel in the middle 
of pin has the same sound as the vowel in the middle of sit. Now you do Example 2 
in the box. Listen to thk key word- and then fill in the oval next to the word that 
has the same vowel sound in the middle as the word I say. The key word is cajge 
(Pause) cape . 

(Pause :07) . 
Did you fill in the oval next to rate? You should have filled in the oval next to 
rate because the vowel in the, middle of rate has the same sound as the vowel in the 
middle of cape . If you did not fill in the oval next to rate, please $o so now. 

(Pause :10) 

Now we will do Parts A and B in the same way. Yoy listen to the key word and 
then fill in the oval next to the word that has/ the same vowel sound in the middle^ 
as' the, word I say. Ready? 

A. The key word is lob (Pause) .lob. 



B. The key word is might (Pause) might , 



NOTE 

The additional J.8 items were presented 
in the remainder of the booklet. 
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Attachment lb 
MINNESOTA TEACHER OUTCOMES STUDY 
Results by Sample and Overall 
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Thirty teachers were selected from all teachers who had completed 
the original booklet, based on the approximate "frequencies of each type 
of special arid regular classroom teacher. The teachers selected were 
of the following types: 



Remedial Reading 

SLBP 

Title I 

Beading Resource 
Supplementary Reading; 
SLD Reading Program 
Classroom Teachers - 3rd Grade 
Classroom Teachers - 3rd/4th Grade 
Classroom Teachers - 4th Grade 



2 
3 
2 
1 
1 
1 



teachers 
teachers 
teachers 
teacher 
teacher - 
teacher 
10 teachers 
1 teacher 
9 teachers 



•Each teacher was mailed a short letter signed by John Adams, the 
questionnaire (see Attachment A), the Teacher Outcomes Booklet ,. the 
original cover letters, and a return envelope. They were asked to 
complete the questionnaire as soon as possible and return it to RTI. 
No identification code was included. 



is 
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tr nno a U Deportment of Education 

msf P' :*3? , T l ' n *-' ra £&ii5 1 l J Capitol Square. 550 Cedar Street 

i 



M St. Paul, Minnesota p5101 

i 
/ 

/ 

/ 



/ 



Dear Teacher: 



In February of this year, you completed a Teacher Outcomes Booklet in 
connection with the Minnesota Statewide Educational Assessment Prpgram. 
This is the first time this instrumert has been administered, and we are 
now jtrying to assess its value and to determine ways in which it can be 
improved. 



I would appreciate your answering the attached questionnaire and returning 
it in the self-addressed envelope* as .soon as possible. A copy of the 
bodklet and cover materials are enclosed for your reference. Please re- 
turn them^with the questionnaire. / 

Your name will not be associated with your responses— it appears only 
on the envelope in which you received this letter. 

If you have any questions, please feel free to Contact: 

% / 

Ms. Muriel Elliott / 

Research Triangle Institute 

Research Triangle Park, North Carolina 27709 

(919) 549-8311 ' 

We appreciate your continued cooperation with this project. 
Sincerely, 



John W. Adams, Director 
State Educational Assessment 



JWA:jb 

Enclosures: 1) Teacher Questionnaire 

2) Teacher (nitcomes Booklet 

3) Original cover letters 

/ 
/ 
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MINNESOTA STATEWIDE EDUCATIONAL ASSESSMENT PROGRAM 
TEACIIER QUESTIONNAIRE* 
Teacher Outcomes Booklet 



A. How many undergraduate courses did you take in reading? 



B. How many In-service reading courses or reading workshops have you attended? 



C. (If applicable) How many graduate courses, have you taken in reading? 

D. What teaching experience have you had? 
(~~| First Year of Teaching 

[T 1-3 Years of Teaching 

| \ A+ Years of Teaching 

t 

There are some questions about the Teacher Outymes Booklet below. We would 
appreciate any comments which you wish to make on any. of the questions or on 
areas which you feel are not covered. ^ 

1. Did you find the format of the Teacher Outcomes Booklet easy to follow? 
□ Yes O*o 



Comments: 



2. Approximately how long did it take you to complete the booklet? 
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3. Did you understand the definitions of the following? 

"Minimal. Acceptable Outcome' □ Yes □ No 

"Desired Outcome" □ Yes □ No 

"Predicted Outcome" □ Yes □ No 



Comments: 



Did you feeL-comfortable making judgments about statewide performance? 
□ Yes □ No. 

Comments: \ — , — — - — ; 



5. Please rate how useful you think these results can be when compared to 
the performance results for Minnesota 9-year-olds. 

□ Very Useful □ Somewhat Useful □ Useless 
Comments: ■ 



6. If you have any additional thoughts about the Teacher Outcomes Booklet, 
please comment on them below. 



\ 



Id-4 



* 



Results From: 
MINNESOTA STATEWIDE EDUCATIONAL ASSESSMENT PROGRAM 
TEACHER QUESTIONNAIRE 
Teacher Outcomes Booklet 

How many undergraduate courses did you take in reading? 

Category Frequency 

0 1 

1 5 

2 9 

3 5 

4 2 > 

5 1 - — 
? 1 

» 

How oany-in=service reading courses or reading workshops have you attended? 



Category 


Frequency 


I 0 


8 


1 


4 


2 


1 


A 


'2 


5 


2 


6-8 


1 


8 


1 


10 


1 


12 


1 


18 


1 


20 


1 


? 


1 



(If applicable) How many graduate courses have you taken in reading? 



Category Frequency 

0 6 

1 4 

2 6. 

3 1 

4 1 
6 2 

"Some" 1 

? 3 - 



39' 

Id-5 



What teaching experience have you had? 



CatoRory 
1 

1-3 
4+ 



Frequency 

1 
5 
18 



1. Did you find^the Vormat of the Teacher Outcomes Booklet easy to follottl 
22 □ Yes 1 □ . No 1 □ Somewhjat 



No... "I found it to be confusing." 

\ 1 



\ 



Yes.. "Minimal and predictable outcomes may vary among. teachers , and 
they may change depending on pupils." 



"I was confused as to the criteria we were to use. Gates Reading 
tests or basic texts (they vary in difficulty)." 

"But I feel that I ha'd. to do. too much guessing on the answers." 
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Approximately how long did it take you to complete the booklet? 



Category • Frequency 

1/2 2 

1/2+ 1 

1/2 t 3/4 4 

3/4 . 3 

1 7 
1 1/2 2 

2 3 

? x 2 



Did you understand the definitions of the* following? 
"Minimal Acceptable Outcome" . □ Yes Q No 

"Desired Outcome" □ Yes* [H No 

"Predicted putcome" Yes □ No 

Category / Frequency 
yes-yes-yes 22 



yes-no-yes 1 " _ 

NR 1 

* 

No... "The desired outcome is a very vague judgment to make." 



Yes (to all three) 

"I have been in this school 20 years and 7 years in schools 
nearby, so can't judge statewide performances'. This is a rural 
underprivileged area." (This answer was repeated for Q; **r) — 

"I understand what I expect but I don't know how that compares 
with others." 

"I understood the definitions, but found them confusing to work 
with." 

•"Terminology could have been simplified. 11 
"Easily defined by example." 

"I understood the definitions, but found making these judgments 
difficult. 11 

l! I had to refer tc the definitions constantly, however." 
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4. Did you feel comfortable making judgments about statewide performance? 
4H Yes 20 □ No 

' " .1 . - " 

ThiTbackgrounds of the three teachers who responded "jtes are: 



\ 



ERLC 



Undergrad 
Courses 

3 

2 

4 
0 



• 


Grad 


Years 


Workshops 


Courses 


Teaching 


4 


2 


4+ 


20 


6 


4+(27) 


8 


2 


4+ • 


0 


? 


4+(14) 



com. lent 



No... "I feel it's unfair to generalize statewide. These judgments 
would vary according to your community . 11 

"I wondered if I underestimated— however , I feel these questions 
would be quite difficult for 9-yea?-old children to answer 
^ correctly. 11 • 

"For my own particular classroom, there would be more accuracy. 
Some of the state's schools do not require so much as others do!" 

"I hctve worked with first and second graders in the classroom and 
small groups in a tutor position. I did not feel qualified to 
make judgments on 9 year olds." 

"I did not feel qualified to make these judgments." 

"It depends so much on the home environment. I find it varies 
even from year to year." 

i 

"By only living in the state for 1 1/2 years, it was difficult 
for me to make judgments." 

"I feel these are so nebulous hard to be definite." 



Yes.. "Have taught in various sections of Minnesota, so I feel that 
through these experiences I can judge 9 year old performance 
relatively well." 

"After 27 years of experience with a Master 1 s Degree in Reading, 
I do feel comfortable making judgments. V 
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Please ,rate how useful you think these results can be when compared to 
the performance results for- Minnesota 9-y ear-olds. f 

7 □ Very Useful 17 -Q Somewhat Useful 0 □ Useless 



Somewhat Useful 

"I'm not sure what the results are going to be used for." 

"When a person working daily with these children completes the 
booklet •" (dash was on the paper) 

"Because my fellow third-grade teachers and I were in agreement 
that we did a lot of guessing, I'm dubious." 

"Though I have no idea how." 



Very Useful 

<< 

"They can be very useful — depends on how accurate my estimations 
are and if and how they are used*" 

"If used with the attitude that each child is an individual 
and has day to day variation in learning." 
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If you have any additional 
please comment on them ~ 



thoughts about the Teacher Outcomes Booklet , 



below. 



Although almost all of tile teachers found/the format easy to follow 
and, the definitions easy 7 to understand, *ery few (3) felt comfdrtable 
estimating statewide performance. /, _ 

/ / I 

"I tried to do'my best but when I had finished.! felt that my 
estimates wex^ probably quite^ inaccurate. 11 j j 

"I thought^ tern 3 was quite difficult for/ttie average stjdent." 

"Large variance between' small schools, small class enrollments 
with lots of independent help compared with "open 11 classrooms, 
large classrooms, Wer-city classes. It f s hard to compare 
all types." 

"I think the reading series usee/ in each school will cause a 
variation in outpomes on a test like this." 

"The test itself seems too wordy—too much reading to do for a 
fourth grade child. Also, many children at the time of the 
test were already 10 years old. The nine year olds tested were 
often the more immature children." 

"I feel we have to take many factors into consideration when we 
judge abilities of human beings. It will give us a feeling of 
achievement when our particular school does well but it can also 
have the opposite affect." 

"I am eager to see the results of teacher 'judgment compared to 
actual 9 year old 1 performance." 

"Some of the items seemed too involved for 9 year olds. I have 
the feeling that either the selections were quite easy or they 
were rather complicated or confusing as stated (elg. p. 1*0. I 
thought that the booklet was well done and I was curious as to 
ihe source of the selections." 

"Probably a waste of time and money." 

"I believe this would be more valid if it were completed by those 
working with 9 year olds." 

"I be3ieve I did a lot of guessing. Therefore, when I completed 
the booklet, I had a feeling of guilt. I like to be f certain 1 
about my answers." 

"I would think this might be more useful when just based on 
your local school district or area." 
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TRODUCTION 



This booklet contains some of the reading items which are being administered 
to a sample .of 9-ryear-old Maine students during March.;,, For~each- item, 
please indicate £hree percentages relating to Maine 9-year-olds.' First, 
indicate your minimal acceptable outcome or the percent of Maine 9-year-olds 
-you-believe must be able to respond correctly to the item in order for you 
to consider that reading- instruction in the state meets the most basic needs 
of these students. Second, indicate your desired outcome or the percent of 
Maine 9-year-olds you would like to see respond correctly to the item. 
Finally, please indicate your predicted outcome or the percent of Maine 
9-year-olds you believe will actually respond to the item correctly. 



ERIC— 



Here are some questions you may have as you prepare for this task. The answers 
are designed to help you estimate the percentages. 



1 # How is a 9-year-old defined for this assessment? 

A 9-year-old is a student born during the calendar year 
1964. Approximately three-fourths of them will be in 
P the fourth grade, and approximately one-fourth of them 

will be in the third grade. A few may be in the second 
pr fifth grades. 
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2. If I don't feel comfortable estimating state percentages, sh 
my percentages "upon my class or students in my_,school? 

You should* attempt to base your per cents upon students 
throughout the state of Maine. The degree to which 
you will feel comfortable doing this will depend upon • 
your training and experience. In practice you will have 
to relate to your own experiences with students. Please 
generalize to the state population as much as your expe- 
rience will allow. 

The -following may help you in determining percehts. 
Suppose 30 9-year-olds represented the total population 
of 9-year-olds in Maine. If so, the chart below would 
hold true: * 

Number of Percent of 

Students Students 

30 100% 

27 90* 
24 80% 
21 70S! 




\ 



18 60% 

15, 50% 

12 40% 

9 30Z 

6 20%- 

3 10% 



iept 



3. What is meant by minimal ac&rptable outcome? 

This is the percent of Maine 9-year-olds you believe 
must be able to respond correctly to a particular reading 
item" in order for you to consider reading instruction to 
be providing essential reading skills to these students. 
To determine 'the minimal acceptable outcome most People 
will probably^consider the importance of the material being 
measured by the item in terms of mastering future reading 
skills for reading at a level necessary to operate success- 
fully in society. 



ERIC \ ' -. * 48 



IIa-3 





The minimal acceptable outcomemight be viewed in another 
way. If the percent of Maine 9-year-olds responding 
correctly to the item were below this percent, you would 
feel very concerned about the instruction of the reading 
skill measured by that itenw 



What is meant by desired outcome ? ^ ^— 

• 

This is the percent of Maine 9-year-olds you believe 
should be able to respond correctly to a particular reading 
item. If reading instruction were optimal for Maine 
Students , this is the percent that would respond correctly 
to the particular -exercise. To determine desired outcome, 
most people will probably consider the importance of the 
material being measured by the item tfs well as the difficulty 
of the item in terms of the general abilities of 9-year-olds. 

The desired outcome might be viewed in another way. If the 
percent of Maine 9-year-olds responding correctly to the item 
were above this percent, you would feel rather satisfied about 
the instruction of the reading skill measured by that item. 



What is meant by predicted outcome? 

This is the percent of Maine 9-year-olds you believe will 
respond\correctly to a particular reading item. To determine 
predicted-outcome, most people will consider the level of 
reading instruction in the state, as they perceive it, along 
with the difficulty of the item in terms of both the general 
abilities of 9-year-olds and the ability of 9-year-olds to 
master the particular item with present reading instruction. 



Can a predicted outcome ever be higher than a desired out/cpme? 

Yes. For instance, you may believe present reading instruction 
emphasizes mastery of certain skills which are of minor 
importance to good reading instruction. If so, you may 
feel that the actual outcome will be better than your desired 
outcome on the particular item. 
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The next few pages contain" some of the reading items which will be administered 
to 9-year-olds in Minnesota. Examples used to prepare the students to respond 
are included with each item. The tape scripts, whichNshow what was read to the, 
students on audio tape as they re«fe the item to themselves, are also included. 
For each item, students were given an amount of time considered adequate for most 
students to respond without time pressure. • , 

For each item, .there is a place to Indicate your estimates of the minimal accept- 
able outcome, desired outcome, and predicted outcome. Please Place, an in the 
box representing your estimate of each percent , to the nearest 10% . 



\ EXAMPLE 

The boxes you are to complete for each item are shown below. In this example, 
the estimate for the minimal, acceptable outcome is 40%. If the actual outcome 
were below 40%, the teacher would feel very concerned about instruction of the 
reading skill measured by' the particular Item. The estimate for the desired 
outcome is 70%. If the actual outcome were above 70%, the teacher would, feel 
rather satisfied about instruction of the reading skill measured by the particular 
item. In this example, the estimate for the predicted outcome is 60%. The 
teacher believes 60% of Minnesota 9-year-olds will actually respond to the partic- 
ular item correctly. 
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ITEM 1 AS PRINTED 
Rdad the two storiesW answer the question which follows them. 



) 



Story 1 

.A^aTiHsome prince was riding his horse in the woods. He saw a dragon chasing a beautiful 
princess. The prince killed the dragon. The prince and the princess were then married. 



Story 2 

Mary was taking a boat ride on a lake. The boat tipped over. Mary was about to drown 
when a young man jumped in the lake and saved her. 



f Story 2 ends like Story 1, what would happen next in Story 2? 



• CD A prince would kill a dragon, 
i o The young man would become a prince. 
» CD Mary and the young man would get married. 
« CD The king would give the young man some money. 



• CD I don't know. 



Indicate, yuivi utmatu ion. Itzm 1 hunt: 



PERCENT 


10 


AO 


30 


40 


50 


60 


70- 


SO 


90 


100 


HuiunaL 
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ITEM 2 AS P RINTED £"N 




. 2. Read the story and answer the question which follows it. 



One day Amos the Ant took his lunch to the park. He sat under a tree and started to eat. 
Then some children came over. Amos gave them some food. It was a-firte day for a picnic. 



What did Amos do FIRST in the story? 

• o He had a picnic, 
t o He ate his lunch. 

% O He climbed a tree. 
« O He went to the park, 
i CZD He found some children. 

• o I don't know. , f 

'/ 

1 Indicate you/i utimatu boh. Item 2 fie/te: 



PERCENT 


10 


20 


30 


40 


50 


60 


70 


no 


90 


100 


ULnimal 
VuVizd 
VfitdicJtzd 
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• ITEM 3 AS PRINTED ^ 
3« Read the passage <and answer the question which follows it* 

One spring Farmer Brown had an unusually good field of wheat. Whenever he saw any birds 
in this field, he got his gun and shot as many of them as he could. In the middle of the summer 
he found that his wheat was being ruined by iqsects. With no birds to feed on them, the insects 
had multiplied very fast; What Farmer Brown did not understand was this: A bird is not simply 
an animal that eats food the farmer may want for himself. Instead, it is one of many links in the 
complex surroundings, or environment , in which we live. 

How much grain a farmer can raise on an acre of ground depends on many*factors. All of 
these factors can be divided into two big groups. Such things as the richness of the soil, the amount 
of rainfall, the amount of sunlight, and the temperature belong together in one of these groups. 
This group may be called non-living factors. The second group may be called living factors . The % 
living factors in any plant's environment are animals and other plants. Wheat, for example, may 
be damaged by wheat rust, a tiny plant that feeds on wheat; or it may be eaten by plant-eating 
animals such as birds or grasshoppers. 

It is easy to see that the relations of plants and animals to their environment are very complex, 
and that any change in the environment is likely to bring about a whole ser ies of chan ges. 



What is the MAIN idea of this passage? 

• o Farmers should not shoot any birds. 

t gz> Insects eat up all the farmer's crops. 

% cd No crops can be grown without sunlight. 

« CZ> Birds eat up most of the farmer's grain. 

» CD Altlmng things are affected by living things. 

I don't know. 



Indicate, yoaJt tetimcutu ion Itm 3 hvtit 



PERCENT 


10 


20 


30 


40 


SO 


60 


70 


80 


90 


JJ0JL 


ttinvnaZ 
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ITEM A AS PRINTED 

Number the events in the order in which they would\j©en at a baseball game. Place a 1 in the 
box beside the event that would happen first. Place a TnTthetox beside the event that would 
happen next. Continue to number the events in the order in which they would happen at a 
baseballgame. 

. [ _ { • The final score was Yankees 5, Red Sox 4. 


[ [ The home team was taking the field for the start of the game. 

1 I The P*°Pfe m tne stands stood for the seventn inning stretch. 

| | The crowd cheered as the third out in the fourth inning was made. * 

The Yankee outfielder slid into home plate, tying the ball game in 
the ninth inning. 



I don't know. 



SCORING INFORMATION FOR ITEM 4 
The order of events hasfto be exact to have a correct response. 



Indicate youA zt>tim<vtte ioK ltm-4 hint: 



PERCEMT 


W 


20 


30 


40 


50 


60 


70 


to 


90 


100 


iiaUmaZ 
Vn.idlcX.zd 




















i 
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NOTE 



The additional IX items were presented 
in Cfie>remainder of the booklet. 
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Attachment lib 
MAINE TEACHER OUTCOMES STUDY 
Results by Sample and Overall 
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Maine Teacher Outcomes Study 
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Item 


Sampl 


e 1 . 


Sample 2 


Sample 3 


Overall 




X 


SE 


X 


SE 


X 


SE 


% 


SE 


1 


Minimal 
Desired 
Predicted 


. 54.2 
82.4 
69.2 


2.16 
2.12 
1.46 


50.7 
79.1 
65.1 


1.47 
1.64 
1.97 


49.0 
80.8 
66.0 


1.48 
1.59 
1.15 


51.3 
80.8 
66.8 


1.00 
1.04 
0.90 


z 


Minimal 
Desired 
Predicted 


58.7 
86.9 
72.6 


2.33 
1.94 
2.28 


54.2 
80.2 
67.9 


1.98 
1.98 
2.86 


52.4 

80.2, 

68.0 


1.20 
1.88 
1.85 


55.1 
82.4 
69.5 


1.10 
*1.12 
1.37 


3 


Minimal 
Desired 
Pred icted 


36.3 
63.1 
46.0 


2.22 
1.72 - 
2.34 


32.8 
57.0 
41.0 


1.79 
2.80 
2.51 


34.0 
62.8 
46.1 


2.02 
2.16 
2.08 


34.4 . 

61.0 

44.4 


1.16 
1.31 
1.34 


4 


Minimal t 

Desired 

Predicted 


48.6 
76.5 
60.9 


1.69 
1.71 
1.50 


44.3 
71.4 
56.4 


2.33 
2.74 
2.71 


41.9 
69.6 
52.0 


1.43 
1.97 
2.15 


44.9 
72.5 
56.4 


1.07 
1.26 
1.26 

- V 


5 


Minimal 
Desired 
Predicted 


54.9 
83.5 
69.6 

•• 


2.42 
1.48 
2.30 


55.5 
81.5 
69.0 


2.03 
2.04 
2.02 


52.6 
79.7 
66.8 


2.48 
2.45 
2.49 


54.4 
81.6 
68.5 


1.34 
1.17 
1.32 


6 


Minimal 
Desired 
Predicted 


52.8 
W7.9 
66.8 


1.46 
1.46 
1.76 


49.3. 

75.3 

60.4 


2.65 
2.64 
2.64 


49.3 
76.1 
60.1 


1.56 
1.52 
2.06 


50.5 
76.5 
62.4 


1.14 
1.13 
1.26 


7A 


Minimal 
Desired 
Predicted 


57.8 
86.3 
71.4 


1.95 
1.18 
2.13 


58.5 
85.0 
71.8 


1.73 
1.79 
1.90 


55.4 
83.6 
72.8 


1.68 
1.71 
1.49 


57.2 
85.0 
72.0 


1.03 
0.91 
1.07 


7B 


Minimal 
Desired 
Predicted 


55.7 
81.6 
67.0 


1.42 
1.39 
1.98 


53.9 
78.9 
65.2 


2.23 
1.32 
1.60 


52.2 
78.0 
66.3 


1.91 
1.49 
2.50 


53.9 
79.5 
66.2 


1.09 
0.81 
1. 19. 


8 

* 


Minimal 
..Desired 
Predicted 


47.2 
75.5 
. 59.9 


1.98 
1.67 
2.26 


44.0 
70.2 
54.7 


1.78 
1.81 
2.01 


"45.4 

72.38 

57.0 


2.00 
2.41 
2.85 


45.6 
72.7 
57.2 


1.11 
1.15 
1.38 


9A 


Minimal 
Desired 
Predicted 


>■■■■■■■. 

50.6 
79.5 
64.0 


2.56 
2.08 
2.73 


50.9 
77.6 

A^ 9 
O 3 • Z 


1.61 
1.55 

1 97 
X • Z / 


48.7 
77.2 

ft 


1.96 
2.51 
9 7? 

£ * 1 i. 


50.1 
78.1 

UJ . u 


1.2Q_, 
1.20 
1 ^5 

X * J J 


9B 


Minimal 
Desired 
Predicted 


43.9 
72.6 
56.5 


2.21 
1.78 
1.60 


42.5 
69.6 
53.8 


1.98 
2.19 
•1.76 


43.5 
71.6 
57.0 


2.46 
2.73 
2.56 


43.3 
71.3 
55.8 


1.28 
1.31 
1.16 




* 






r 
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Maine Teacher Outcomes Study (con.) 



Item 



Sample 1 



SE 



Sample 2 



Sample 3 



SE 



SE 



Overall 



SE 



10 


Minimal 
Desired 
Predicted 


56.9 
84.4 
70.2 


2.31 
1.42 
1.39 


54.5 
81.9 
67.6 


2.24 
1 .82 
2.46 


53.5 
81.6 
66.8 . 


1.87 
1.66 
. 2.28 


55.0 
82. 6 
'68.2 


1.24 
0. 95 
1.21 


11 


Minimal 
Desired 
Predicted 


55.3 
8.2.2 
67.9 


1.94 
2.02 
2.19 


55.9 
82.4 
68.7 


1.86 
1.66 
1.86 


50.9 
79.4 
65.6 


1.95 
1.63 
2.87 


54.0 
81.3 
67 .4 


1.11 
1.03 
1 . 3d 

• 


12 


Minimal 
Desired 
Pred icted 


59.1 
85.3 
73.1 


2.52 
1.55 
1.92 


57.0 
83.6 
71.6 


1.93 
1.49 
1.57 


55.9 
82.9 
71.6 


2.56 
1.15 
2.12 


*7.3 
83.9 
72.1 


1.36 
0.81 
1.09 


13 


Minimal 
Desired 
Predicted 


43.0 
74.0 
55.5 


2.23 
2.19 
1.83 


• 38>2 
65.3 
48.8 


3.07 
2.93 
3.18 


39.0 
67.8 
50.1 


2.77 
2.59 
3.05 


40.1 
69.1 
51.5 


1.57 
1.49 
1.59 


Theme 
6 


Minimal 
Desired 
Predicted 


51.4 
?8.7 
64.3 


1..57 

l.OK 

1.23 


49.2 
75.4 
N61.3 


1.43 
1.29 
1,50 


47.9 
75.6 
61.6 


1.12 
1.30 
1.59 


49.5 
76.5 
62.4- 


0.80 
0.70 
0.84 


Theme 

7 


Minimal 
Desired 
Predicted 


51.5 
79.7 
64.6 


1.96 
1.62 
1.54 


49.9 
76.8 

\62.3 


1.53 
1.32 
1.24 


48.6 
76.8 
62.5 


2.41 
1.45 
1.35 


50.0 
77.7 « 
63.1 


1.09 
0.94 
1.04 


Total 


\- 

Minimal 
Desired 
Predicted 


51.4 
79.1 
64.4 


1.65 
1.22 
1.31 


49\ 
76. 0\ 
61.8 \ 
— m. 


1.32 
1.14 
1.24 


48.2 
76.1 
61.9 


1.86 
1.83 
1.67 


49.7 
77.0 
62.7 


0.85 
0.72 
0.86 
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Overall Estimates with 
Actual Student Performance 




Maine Teachfer Outcomes Study 



Item 



Teachers Estimates 



Minimal 



Desired 



Predicted 



Student 
Performance 





% 


SE 


% 


SE 


% 


SE 


Z 


SE 


1 


51.3 


1.00 


80.8 


1.04 


66.8 


0.90 


73.0 


1.99 


2 


55.1 


1.10 


82.4 


1.12 


69.5 


1.37 


81.5 


1.63 


3 


34.4 


1.16 


61.0 


1.31 


44.4 


1:34 


32.2 


2.15 


4 


44.9 


1.07 


72.5 


1.26 


56.4 


1.26 


40.3 


2.43 


5 


54.4 


1.34 • 


81.6 


1.17 


68.5 


1.32 


84.8 


> 1.07 


6 


50.5 


1.14 


76.5 


1.13 


62.4 


1.26 


59.9 


2.30 


7A 


57.2 


1.03 


85.0 


0.91 


72.0 


1.07 


75.3 


1.68 


7B 


53.9 


1.09 


79.5 


0.81 


66.2 


1.19 


52.0 


1.82 


8 


45.6 


1.11 


72.7 


1.15 


57.2 


1.38 


68.8 


2.02 


OA 


SO 1 


1.20 


78.1 


1.20 


63.6 


1.35 


68.3 


„2.27 


on 






71.3 


1.31 


55.8 


1.16 


62.1 


2.26 


10 


55.0 


1.24 


82.6 


0;95 


68.2 


1.21 


74.3 


2.10 


11 


54.0 


1.11 


81.3 


1.03 


67.4 


1.35 


86.4 


1.49 


12 


57.3 


1.36 


83.9 


0.81 


72.1 


1.09 


67*8 


1.83 


13 


40.1 


1.57 


69.1 


1.49 


31.5 


1.59 

i 


35.7 


2.25 


Theme 
6 


49.5 


# ■ 

0.80 


76.5 


0.70 


♦ 

62.4 


0.84 


63.1 


/ 

0.87 


Theme 
7 


50.0 


1.09 


77.7 


0.94 


63.1 


1.04 


65.8 


1.15 


Total 


49.7 


0.85 


77.0 


0.72 


62.7 


0.86 


64.2 


0.87' 
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MAINE TEACHER OUTCOMES STUDY 
f Opinion Survey 
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Twenty-five teachers were selected from all teachers who had 
completed the original booklet, based on the .approximate frequencies 
of each type of special and regular classroom teacher. The types of 
teachers selected werei 

Remedial Reading 3 teachers 

SLD Reading Program 1 teacher 

Reading Consultant 1 teacher 

Reading Supervisor 1 teacher 

Developmental Reading 1 teacher 

Classroom Teachers - 3rd Grade 9 teachers 

Classroom Teachers - 3rd/4th Grade 1 teacher 

Classroom Teachers - 4th Grade 8 teachers 

Each teacher was mailed a short letter signed by Dr. Maxcy, the 
questionnaire (see Attachment A), the Teacher Outcomes .Booklet , the 
original cover letters, and a return envelope. ■ They were asked to 
complete the questionnaire as soon as possible and return it to RTI. 
No identification code was included. * 
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STATt OF MAINE 



Department of 
Educational and Cultural Services 



AUGUSTA. MAINE 04330 



Dear Teacher: 

In March or April of this year, you completed a Teacher Outcomes Stu 
Booklet in connection with the Maine Assessment of Educational Progress. 
This is the first time this instrument has been administered* and we are 
now trying- to assess its value and to determine ways in which it can be 
improved • 

m 

I would appreciate your answering the attached questionnaire and re- 
turning it in the self-addressed envelope as soon as possible. A copy of 
the booklet and cover materials are enclosed for your reference. Please 
return them with the questionnaire. 

Your name .will not be associated with your responses — it appears onl] 
on the envelope in which you received this letter. 

If you have any questions, please feel free to contact: 



Ms. Muriel Elliott • 

Research Triangle Institute 

Research Triangle Park, North Carolina 27709 

(919) 549-8311 




We appreciate your continued cooperation with this project. 



Sincerely, 



Horace P* >Iaxcy, Jr. 
Coordinator, State Educational 
Assessment Program 



HPM:lh. 




^ Enclosures: 1} Teacher Questionnaire 
W. 2) Teacher Outcomes Study Booklet 

^ & 3) Original cover letters 

Y'Z *) Return envelope 
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MAINE ASSESSMENT OF EDUCATIONAL PROGRESS 
TEACHER QUESTIONNAIRE 
Teacher Outcomes Study Booklet ^ 
- " V 

A. How many undergraduate courses did yo$ take in reading? 

* • \ 

_ — ^ 

B. How many in-service reading courses or reading workshops have you attended? 



C- (If applicable) How many graduate courses have you taken in reading? 



D. What teaching experience have you had? 

j | First Year of Teachin'g 

I | 1 - 3 Years of Teaching 

i 

|"~] A+ Years of Teaching 



There are some questions about the Teacher Outcomes Study Booklet below We 
would appreciate any comments which you wish to make on any of the questions 
or on areas which you feel are not covered. 

1. Did you find' the format of the "minimal/desired/predicted" sections of the 
Teacher Outcomes Study Booklet easy to follow? 

□ Yes • □ No 

Comments : , — — ~ ' 



2. Did you find the format of ' the section on stimulus materials easy to follow? 
, □ Yes 13 Nq 



Comments: 
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Approximately how long did it take you to complete the booklet? 

— \ 

Did you understand the definitions of the following? 



"Minimal Acceptable Outcome" 


□ 


Yes 


□ 


No 


"Desired Outcome" 


□ 


Yes 


a 


No 


"Predicted Outcome" 


□ 


Yes 


■ □ 


No 



Comments : 



Did you feel comfortable making judgments about statewide performance 
□ Yes □ No 

Comment s : 



Please rate how useful you think these results can be when compared t« 
the performance results for Maine 9-year-olds. 

□ Very Useful □ Somewhat Useful □ Useless 
Commen t s : • : . — — 



If you have any additional thoughts about the Teacher Outcomes Study 
Booklet , please . comment on them below. 

— / ■ * — - 
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Results Prom: S 
MAINE ASSESSMENT OF EDUCATIONAL PROGRESS 
> TEAQ1ER OPINION QUESTIONNAIRE 
Teacher Outcomes Study Booklet 



How many undergraduate courses did you take in readin 



Category 



0 
1 

1-2 

2 

3 \ 
4 

4-5 
6 
8 
? 



Frequency 

1 
■2 
1 
1 
4 
2 
1 
1 

. 1 
1 



How many in-service reading courses or reading workshops have you 
attended? 



Category 

0 
1 
2 
3 

5+ 
5-6 
10 
11 
several 
? 



Frequency 
2 

3 _ 
3 

1 . 
2 



(If applicable) HowNnany graduate courses have you taken in reading? 
Category Frequency „ 



0 
1 

2-3 
3 
4 

4-5 
5+ 



8 
3 
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D. What teaching experience have you had? 
2 □ First Year of Teaching 
2 Q 1-3 Years of Teaching 
12 Q 4+ Years of Teaching 



1. Did youttnd the format of the "minimal/desired/predicted" sections 
' * 0 f the Teacher Outcomes Study Booklet easy to follow? 

12 Q Yes No 



No... "Most of the time, the difference was easy to determine, but 
on some questions the outcomes were hard to determine. 



Yes.. "Almost too easy. I found that I tended to cluster my responses 
—when in^dpubt J I changed my answers to avoid clustering- 
Blanks to be filled in might be used. Ex. : minimal 40 

desired 70_ 
• predicted 60_ " , 

"It's a good thing you explained each fully." 

®. "Items 1-8 M.I. Yes;~item 9 Yes; Item 10. Yes; Item 11 No (Idea is 
directly stated.); Item 12 Yes; Item 13 Yes." 

"This would follow naturally within class sessions." 

"I feel it is very difficult to predict this on a statewide basi: 



-ERLC- 
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2. Did you find the format of the section on stimulus materials easy 
to follow? 



13 □ Yes 



3 □ No 



Yes*. "Some pupils might be confused by the numbers used in the selection 
RAINING." ' 

"This would follow naturally within class sessions." 

"It is difficult to say whether a particular passage is appropriate 
or inappropraite because children differ in interests and backgrounds 
What is appropriate for one is not necessarily appropriate for all." 



Ql and Q2: One teacher responded that he/she didn't understand 
the minimal/desired/predicted sections but did understand the 
section on stimulus materials. This teacher was in his/her 
first year of teaching, has had 3 undergraduate reading courses, 
and had attended one workshop. 

Three teachers responded that they understood neither section* All 
three had been teaching for more than 4 years and had the following 
backgrounds; 



3. Approximately how long did it take you to complete the booklet? 



Undergrad 
Courses 



Workshops 



Grad 
Courses 



4 
1 
? 



Several 
1 
? 



0 
1 
4-5 



Category 
1/2 - 2/3 



3/4 



Frequency 

1 
3 




til 



.67 



IId-7 



0 

ERIC 



4. Did you understand the definitions of the following? 
"Minimal Acceptable Outcome □ Yes □ No 

"Desired Outcome" □ Yes O No 

"Predicted Outcome" Q Yes □ No 

Category Frequency 
yes-yes-yes 15 



no-no-no 



1 




Yes (to all three) 

, "This" would follow naturally within class sessions." 

"Ideally, shouldn't our minimal acceptable outcome be approximately 
the equal of desired outcome—if we are to teach skills fqr mastery? 

"Predicted was hard to differentiate fr^n Desired for me. \ I found 
that I almost interchanged the two," 

"I understoo<Tthe explanations , but found (1) hard to make some of 
the judgments/ 1 

"Am not accustomed to thinking in these terms, however, and it Was 
•thought pre voking 1 to use this criteria." 



The one teacher who responded that he/she did not understand any 
of, the definitions had had more than 4 years of teaching expedience, 
4 Undergraduate courses in reading, several workshops, and no 
g^duate courses in reading. He/she did not Comment. 



i 

-7 



68 

IId-8 



5. Did you feel comfortable making judgments about statewide performance? 
4 □ Yes 12 □ No 

~No~. .."I'm a first year teacher. " 

"I thought some of these stories were too advanced and vocab. too 
difficult for nine~ year old students." 

"Not particularly since this state (Maine) is large with a great 
variation in its educational, economic and cultured factor's. There 
is little opportunity to compare methods, practices and materials 
with people from other. areas. 11 

"Am concerned about what goes on here and it's hard to think about 
the entire state, knowing many areas have entirely different back- 
grounds." 

"My experience has been in one elementary school for only 3 years." , 

Yes.. "Having worked in various parts of JLhe state I felt that I could 
- judge fairly accurately from the students I had had." 



Only four teachers said that they felt comfortable making statewide 
judgments. Their backgrounds were as follows: 

Undergrad Grad Years 

Courses Workshops Courses Experience 

2 3 0 1-3 no comment on this^ 

3 5+ 5+ — 4+ " 
8 10 14+ 

3 2 2-3 • A+(1A) comment on this Q. 
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6. 



Please rate how useful you think these results can be when cojnpared 
to the performance results for Maine 9^year-olds. 



4 Q Very Useful ' 10 Q Somewhat Useful 



2 Q Useless 



Somewhat Useful: 

"Would want to study the results before I 
I'm skeptical. 11 

"Perhaps if used to ^show the correlations 



termed them very anything. 



between performance by 



students from a certain area and teacher Expectations in that same 



area/ 



Very Useful 



"It would be most interesting to gathe/ information concerning ^ 
materials, methods; organizational patterns, and classroom Prices 
from selected areas throughout the stite and compare their effective- 
ness in reaching the pupils-Cnine-y^ar-olds and then continue 
with other age groups.)." 



"If the results are used to implement and complement the reading 
'program statcwise.' If school districts receive data and use it 
io modify or bolster their Reading program, the survey will have 
been very useful." 



Two teachers thought -the Teacher Outcomes results would be useless, 
Their other responses were as follows 



Under grad 
Courses 

4 
? 



Workshops 

Sever a}. 
? 



Grad 
Courses 

0 
4-5 



Years 
Experience 

4+ 
■ 4+ 



Format Easy to Follow 
M/D/P Stimulus 



no 
no 



no 
no 



Time 

2 Evenings 
1 hr. 



Definitions 



M 



D 



no no no 
yes yes yes 



Judgments 



no 
no 



Of the four teachers who felt that the results would be very useful, 
three also said that Lhey felt comfortable making statewide judgments 
of performance. 



\ 
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In general it seems that the teachers found the format easy to follow 
and that they understood the definitions of minimal, desired, and predicted 
outcomes. However, except for a few tdachers who had had relatively rabre 
experience, the majority felt uncomfortable making judgments on a statewide 
basis. Only two teachers (alt that the study would be useless; these teachers 
had a generally negative opinion of the quesitonnaire as a whole. \^ 



7. If you have any additional thoughts about the Teacher Outcomes Study 
Booklet » please comment^on them below. 



I! A waste of time." 

"Lengthy." ' ^ 

"You should have taken in£o-- j:ons i dera t ion the students that have 
a bilingual problem." ~ jT 

"I think that if I had had opportunity to use the assessment 
materials with a selected group of pupils — from all ranges of 
ability and achievement — I would have felt that my assessment 
of the project would have been more valid. Working as I do 
mainly with pupils of low achievement and/or accompanying 
emotional problems, my judgment may be somewhat slanted." 

"I'd be interested in knowing how I was selected for a second 
response to the final results of this study." 

"I am not sure of the* value of the^survey if there is any. I 
am not impressed by surveys and generalizations." 

"Some were too difficult for 9 year olds." 

"A magnificent effort. I'll be very interested to see the results 
and find out exactly how useful something like this is." 
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Attachment Ilia 
RICHFIELD TEACHER OUTCOMES STUDY 



Results by Item 
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Richfield Teacher Outcomes Study 
9-Year-Olds Package 1 



Teacher Estimates 



Actual 



Teachers' 
Prediction- 





Minimal 






Student 


Student 


Item 


Acceptable 


Desired 


Predicted 


Performance 


Perf or mar 


1A 


— tr — 

40 


80 


60 


75.0 


-15. 0 


IB 


65 


90 


80 


7l%4 


Q £ 

o. o 


1C 


35 


65 


60 


62*3 


— Z. J 


2A 


95 


100 


98 


97*4 


0. o 


2B 


95 


100 


98 


97>o 




2C 


95 


100 


98 


A1 / 

91. 4 


o. o 


2D 


95 


100 


98 


Ar A 

95.8 


2.2 


2E 


90 


98 ' 


95 


on / 

80.4 


14. o 


2F 


95 


100 


98 


93.0 


5.0 


2G 


95 


100 


98 


91.6 




2H 


95 


100 


98 


90.4 


7.o 


3 


50 


75 


60 


73.0 


-13.0 


4A 


65 


90 


75 


65.1 


9.9 


4B 


40 


80 


55 


78.0 


-23. U 


4C 


40 


75 


60 


73.7 


-13. 7 


5A 


40 


75 


60 


85.1 


-25. 1 


5B 


60 


80 


70 


92.4 


-22. 4 


5C 


50 


80 


65 


87.2 


-22. 2 


5D 


~~ 30 


60 


40 


47.8 


-7.8 


5E 


30 


60 


40 


61.0 


-21. 0 


5F 


50 


70 


60 ... 


81.8 


-21. o 


5G 


50 


75 


60 


76.9 


-16.9 


5H 


60 


80 , 


70 


71.5 


-1. 5 


6A 


15 


30 


20 


33.6 


-13.6 


6B . 


25 


55 


35 


56.0 


-21.0 


7A 


30 


50 


40 


t 54.0 


— 14 . 0 


7B 


. AO 


65 


55 


60.8 


C Q 

-5.8 


7C 


40 


60 


50 


54*4 


-4.4 


7D 


30 




35 


38 . 1 


—J. X 


7E 


25 


5 Y 


is 


J/ . O 


-9 8 


7F 


40 


70 


50 


52.4 


-2.4 


7G 


30 


50 


40 


41.2 


-1.2 , 


7H 


25 1 


50 


40 


43.9 


-3.9 


8A 


50 


75 


65 


74.8 


-9.8 


8B 


6a 


80 


70 


71.6 


-1.6 


8C 


60 


80, 


70 




-1.4 


9A 


50 


x 80 
90 


65 


59.9 


5.1 


9B 


75 


85 


77.2 


7.8 


9C 


50 


75 


60 


74.2 


-14.2 


10A 


45 


70 


55 


68.5 


-13-* 5 
= 2.6 


10B 


80 


90 


85 


82.4 


IOC 


45 


60 


50 


55.7 


-5.7 



ERIC 
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Richfield Teacher Outcomes Study 
9-Year-Olds Package 1 (con.) 



Teacher 





Teacher 


Estimates 




Actual 


Prediction- 




Minimal 






Student 


Student 


Item 


Acceptable 


Desired 


Predicted 


Performance 


Performance 


11A 


65 


80 


70 


52.5 


17.5 


11B 


50 


70 


60 


66.8 


-6.8 


11C 


50 


80 


70 


68.2 


1.8 


11D 


50 


70 


60 


70.0 


-10.0 


HE 


50 


65 


60 


73.6 


-13.6 


11F 


50 


75 


.65 


_3(L8_ .. 


34.2 


11G 


60 


80 


75 


67.7 


' 7.3 


11H 


60 


80 


75 


62.7 


12.3 


12A 


55 


70 


65 


66.7 


1.7 


12B 


55 


75 


65 


75.5 


-10.5 


12C 


75 


/80 


80 


72.7 


7.3 


13A 


70 


v 80 


75 


95.7 


-20.7 


13B 


70 


80 


75 


97.8 


-22.8 


13C 


70 


80 


75 


98.4 


-23.4 


13D.. 


60 


70 


65 


67.5 


-2.5 


13E 


70 


80 


75 


94.0 


-19.0 


13F 


75 


90 


85 


94.1 


-9.1 


13G 


75 


90 


85 


99.4 


-14.4 


13H 


70 


* - 80 


75 


94.6 


-19.6 


14A 


75 


' 85 


80 


76.6 


3.4 


14 B 


40 


60 


50 


62.4 


-12.4 


15 


35 


60 


45 


39.3 


5.7 


16 


30 


60 


45 


77.0 


*-32.0 



r ) 
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Richfield Teacher Outcomes Study 
9-Vear-Olds Package 2 













Teacher 




Teacher 


Estimates 




Actual 


Prediction- 


Item— ' 


Minimal 






Student 


Student 


Acceptable 


Desired 


Predicted 


Performance 


Performance 


2A 


25- 


50 


7k 

4/0 


86.8 


-46.8 


2B 


25 


50 


40 


59.6 


-19.6 


2C 


45 


60 


50 . 


75.8 


-25.8 


2D 


30 


50 


40 


69.2 


-19.2 


2E 


45 


70 


60 


78.0 


-18.0 


2F 


35 


60 


50 


69.5 


-19.5 


2G 


35 


60 


50 


77.5 


-27.5 


2H 


40 


60 


50 


72.6 


-22.6 


h 


75 


90 


85 


74.3 


10.7 


5A 


85 


95 


90 


86.6 


3.4 


5B 


85 


95 


90 


88.9 


1.1 


5C 


70 


80 


75 


81.4 


-6.4 


5D 


70 


80 


75 


82.8 


-7.8 


5E 


/ 60 


75 


70 


74.0 


-4.0 


5F . 


/ 55 


70 


4 60 


62.1 


-2.1 


5G / 


70 


80 


75 


75.8 


-0.8 


5H/ 


60' 


75 


70 


74.4 


-4.4 


5 7 


85 


95 


90 


93.1 


-3.1 


5J 


85 


95 


90 


88.3 


1.7 


5K 


85 


95 


90 _ 


88.8 




5L 


75 


90 


80 


85.7 


-5.7 


6 


10 


40 


30 


50.4 


-20.4 


7 


75 


90 


85 


72.2 


.12.8 


9A 


|80 


90 


85 


89.8 


-4.8 


9B 


- 80 


90 


85 


83.8 


1.2 


9C 


80 


90 


85 


93.5 


-8.5 


9D 


80 


90 


85 


78.3 
91.4 


6.7 


9E 


80 


90 


85 


-6.4 


9F 


85 


95 


90 


87.2 


2.8 


9G 


75 


85 


80 


48.5 


31.5 . 


9H 


80 


90 


85 . 


86.5 


-1.5 


10A 


75 


90 


80 


77.7 


2.3 


10B 


85 


95 


90 


72.2 


17.8 


IOC 


70 


85 


80 


73.8 


6.2 


11A 


50 


75 


65 


76.0 


-11.0 


11B 


50 


70 


60 


40.7 


19.3 


11C 


45 


75 , 


60 


75.1 


4 -15.1 


11D 


35 


60 


50 


68.8 


-18.8 


HE 


45 


65 


55 


54.3 


0.7 


11F 


50 


75 


65 


67.2 


-2.2 


11G 


55 


75 


65 


52.5 


12.5 


1111/ 


55 


75 


65 


25.9 


y 39.1 
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Richfield Teacher Outcomes Study 
9-Year-Olds Package 2 (con.) 









Teachers' 




Teacher Estimates 


Actual 


Prediction- 




Minimal 


Student 


Student 


Item 


Acceptable Desired Predicted 


Performance 


Performance 



12A 
12B 
12C 
13A 
13B 
13C 
13D 
13E 
13F 
13G 
13H 
14A 
14B 
14C 
15A 
15B 
16 



20 
20 
20 
85 
80 
80 
95 
85 
85 
85 
95 
20 
20 
.20 
60 
60 
' 80 



50 
50 
50 
95 
90 
90 

100 
95 
95 
95 

100 
50 
50 
50 
85 
80 
90 



30 
30 
30 * 
90 
85 
. 85 
98 
90 
90 
90 
98 
35 
35 

35—, 
70 ! 
70 
85 



35.7 
36.2 
66.2 
73.8 
87.2 
74.5 
85.0 
86.6 
92.5 
90.3 
74.3 
28.5 
37.8 
32. 8 
65.2 
62.1 
62.1 



-5.7 
-6.2 
-36.2 
16.2 
-2.2 
10.5 
13.0 
3.4 
-2.5 
-0.3 
23.7 
6.5 
-2.8 
2.2 
4.8 
7.9 
22.9 



-''items 1A, IB, 1C, 4A, 4B, 4C, 8A, 8B, and 8C are identical to items 
1A, IB, 1C, 4A, 4B, 4C, 8A, 8B, and 8C, respectively, in package 1. 



/ 
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Richfield Teacher Outcomes Study 
1 9-Year-Olds. Package 3 



Teacher Estimates 



Itenr^ 



Minimal 
Acceptable , 



Actual 
Student 



2A 


50 


2B 


50 


2C 


50 


2D 


50 


2E 


60 


2F 


75 


2G 


75 


2H 


60 


3A 


60 


3B 


75 


3C 


75 


5A 


70 


5B 


75 


5C 


70 


5D 


60 


5E 


80 


5F 


70 


5G , 


60 


5H 


.75 


6A 


20 


6B 


30 


6C 


50 


7A 


75 


7B 


-65 


7C 


60 


7D 


70 


7E 


65 


7F 


75 


7G 


75 


7H 


75 


9A 


45 


9lT 


45 


9C 


45 


9D ' 


55 


9E 


45 


9F 


55 


99 


40 


9H 


55 


. 10A 


55 


10B 


70 


IOC 


70 



Desired 1 Predicted > Performance 




Teachers' 
Prediction- 
Student 
Performance 



75 


60 


71.4 


-11.4 


70 


55 


78.1 - 


-23.1 


75 


60 


91.7 


-31.7 


70 


60 




-24.5 


80 


70 


* 90.5 


-20.5 


90 


80 


91.5 


-11.5 


90 


80 


84.8 


-4.8 


80 


70 


68.6 


1.4 


80 


75 


69.4 


5.6 


90 


85 


, 89.6 


-4.6 


95 


85 


90.5 


-5.5 


90 


80 


' 76.6 


3.4 


95 


85 


71.3 


13.7 


85 


80 


'68.9 


11.1 


80 


70 


57.8 


12.2 


90 


85 


74.4 


10.6 


80 


75 


76.2 


-1.2 


75 


65 


66.9 


-1.9 


90 


80 


57.8 


22.2 


65 


45 


51.6 


-6.6 


65 


50 


60.5 


-10.5 


70 


65 


64.7 


0.3 


95 


80 


94.1 


-14.1 


85 


75 


70.4 


4.6 


80 


70 


81.3 


-11.3 


85 


75 


90.3 


-15.3 


80 


70 


{ 60.1 


9.9 


95 


85 


83.0 


2TO~ 


95 


85 


90.2 


-5.2 


95 


85 


88.0 ' 


-3.0 


75 


60 


67.7 


-7.7 


75 


60 


77.2 


-17.2 


75 


60 


58.7 


1.3 


-80 


70 


72.4 


-2.4 


75 


60 


73.8 


-13.8 


80 


70 


81.9 


-11.9 


70 


55 


69.2 


-14.2 


80 


70 


63.1 


6.9 


75 


65 


86.2 


-21.2 


85 


75 


87.6 


-12.6 


85 


75 


90.5 


-15.5 
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Richfield Teacher Outcomes Study 
9-Year-Olds Package 3 (con*) 







1 

1 






Teachers' 




Teacher 


Estimates 




Actual 


Prediction- 




Minimal 






Student 


Student 


Item 


Acceptable 


Desired 


Predicted 


Performance 


Performance 


11A 


55 


75 


70 


50.2 


19.8 


11B 


30 


60 


45 


65.6 


-20.6 


lie 


45 


65 


55 


86.6 


-31.6 


11D 


60 


80 


75 


: 69.5 


5.5 


HE 


80 


90 


85 


86.3 


-1.3 


11F 


40 

c 


60 


55 . 


37.6 


17.4 


11G 


60 


80 


75 


73.7 


1.3 


11H 


60 


75 


70 


67.0 


3.0 


12A 


15 


55 


25 


9.9 


15.1 


12B 


15 


50 


25 


41.8 


-16.8 


12C 


20 


55 


30 


30.4 


-0.4 


13A 


50 


75 


65 


75.3 


-10.3 


13B 


75 


90 


80 


90.7 


-10.7 


13C 


50 


75 


65 


82.0 


-17.0 


14A 


45 


70 


60 


70.6 


-10. O 


1AB 


45 


70 


60 


85.1 


-25.1 


14C 


50 


75 


65 


82.7 


-17.7 


14D 


50 


75 


65 


76.2 


-11 . 2 


14E 


30 


s 


40 


86.1 


-46.1 \ 


14F 


55 




65 


87.4 


<• -22.4 v 


14G 


30 


60 


40 


65.6 


-25.6 


14H 


40 


65 


50 


73.1 


-23.1 


15 


70 


90 


. 75 


82.4 


-7.4 


16 


60 


80 


65 


53.2 


11.8 





litems 1A, IB, 1C , 4A, 4B, 4C, 8A, 8B, and 83 are identical to 
items 1A, IB, 1C, 4A, 4B, AC, 8A, 8B, and 8C, respectively, in package 1. 
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Richfield Teacher Outcomes Study 
13-Year-Olds Package 1 



Teacher Estimates 



Minimal 
Acceptable 



Actual 
Student 



Teachers' 
Prediction- 
Student 



32 

52 

41 

93 

93 

93 

93 

93 

93 

93 

93 

55 

51 

50 

50 

50 

50 

50 

50 

50 

50 

46 

46 

46 

46 

46 

46 

46 

46 

44 

50 

44 

50 

50 

50 

50 

50 

50 

50 

50 

60 

60 

60 



Desired 


Predicted 


Performance 


Perf ortnanc 


80 


. 66 


f\f\ C- 

90.6 


O/. A 


90 


82 


80.3 


1. / 


. 77 


70 


> 73.4 


$ -3.4 


100 


95. 


96.0 


-1.0 


100 


95 


89.0 


o.O 


100 


95 


90.9 


4.1 


• 100 


95 


93.7 


1.3 


100 


95 


84.3 


10.7 


100 


95 


87.4 


7.6 


100 


95 


94.6 


0.4 


100 


95 


89.7 


5.3 


88 


76 


68.7 


7.3 


84 


72 


84.6 


-12.6 


86 


74 


72.0 


2.0 


86 


74 


80.9 


-6.9 


86 


74 


78.9 


-4.9 


86 


74 


\ 66.0 


o.O 


86 


74 


69.1 


4.9 


86 , 


74 


80.3 


-O.J 


86 


74 


60.5 


13. 5 


86 


74 


73.0 


I. U 


82 


61 


62.1 


-1. X 


82 


61 


61.5 


-U.j 


82 


61 


61.0 


U.U 


82 


61 


77.6 


-O.O 


82 


61 


90.3 


on o 

-29.3 


82 


61 


76.2 


-15 ml 


82 


61 


73.7 


-1Z. / 


82 


61 


76.4 


-15.4 * 


80 


64 


59.3 


4.7 


86 


79 


70.2 


8.8 


86 


72 


50.2 


11.8 


90 


70 


96.6 


-16.6 


90 


70 


87.4 


-17.4 


90 


70 


94.6 


-24.6 


90 


70 


92.0 


-22.0 


90 


70 


62.1 


7.9 


90 


70 


87.3 


-17.3 


90 


70 


89.9 


-19.9 


90 


70 


89.3 


-19.3 


95 


75 


92.6 


-17.6 


95 


75 


89.4 


-14.4 


9.5 


75 


90.5 


-15.5 

\ 



IIIa-7 
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Richfield Teacher Outcomes Study 







13-Year-01ds 


Package 1 


(con.) 


■ 

— — « — -- 


i i ■ i i ■ i 










Teachers' 




Teacher Estimates 


Actual 


Prediction- 




Minimal 






Student 


Student 


Item 


Acceptable 


Desired 


Pro,] -f 9\f> a/4 

0 


Performance 


Performance 


9D 


en 
50 


y_> 


/ D 


84.6 ' 


-9.6 


9E 


en 
50 


yj 


7^ 


94.6 


-19.6 


9F 


cn 
60 


9-> 


1 5 


? 91.4 


-16.4 


9G 


60 


95 


7 ^ 
/5 


' 88.6 


-13.6 • '" 


' 9H 


60 


nc 

95 f . 


7 <; 


82.3 


-7.3 


• 10A 


48 


O / ' 

84 


A7 


83.6 


-16.6 


10B 


56 


93 


OA 

oh 


67.1 


16.9 


IOC 


57 


92 


Q /, 

on 


92.6 


^•8.6 


11A 


66 


95 


oh 


96.8 


-12.8 


11B 


60 


o7 


"7 1 

/ J 


84.5 


-11.5 


12 


65 


97 


Q A 

OH 


70.7 


13.3 


13A 


47 


OD 




59.9 


11.1 


13B 


44 


O A 


* AA » 


-^84.2 


-18.2' 


13C 


CO 

DO 


y j 


OX 


90.8 


-9.8 


13D 


76 


1UU 




97.4 


-2.4 


14A 


41 


79 


5y 


60.4 


-1.4 


14B 


40 


/o 


«;ft 

JO 


75.0 

' . 74.2 "> 


-17.0 


14C 


/ c 
45 


oH 


AQ 

oy 


-5.2 


14D ' 


46 


OJ 


7/f 

/ h 


75.0 


-1.0 


. 14E 


% , 41 


✓ oZ 


AA 
Oh 


54.6 


9.4 


15A 


60 


95 


ft! 

px 


79.1 


1.9 


', 15B 


45 


\ 95 


/ 5 


* 91.1 


-16.1 


15C * 

i5D r. 


45 


nc 

9.) 


7C. 


9.4.8 


-19.8 


45 


y5 


7S 


92.0 


-17.0 


15E 


45 


QC 

yj 


7 


97.4 


-22.4 


15F 


45 


95 


7H 

/u 


94.8 


-24.8 


15G 


45 


95 


7 «; 
/5 


84.0 


-9.0 


15H 


45 


nc 

95 


7 «; 
/5 


65.4 


9.6 


16A 


50 


QC 

ob 


AA 
DO 


82.6 


-16.6 


16B 


37 


7i 


«;a 
5o 


51.9 


4.1 


16C 


42 


7o 


A1 
01 


42.1 


18.9 ■ 


17A 


65 


96 


Q7 


90.0 


-3.0 


17B 


a , 


Oft 
y o 


87 


98.0 


-11.0 


17C 


66 ' 


98 


88 


73.5 


14.5 


17D 


64 


94 


82 


56.1 


25.9 


17E 


55 


91 


74 


62.1 


11.9 


17F 


59 


93 • 


78 


77.1 


0.9 


17G 


57 


91 


76 


79.5 


-3.5 


17H 


64 


94 


80 


92.4 


-12.4 


18A 


45 


86 


74 


65.1 


8.9 


18B 


30 


76 


41 


35.6 


5.4 


18C 


43 


86 


70 


69.6 


0.4 


-19A 


54 


-93 


79 


75r4 ~ 


3.6 


19B 


59 


97 


85 


* 91.4 


-6.4 


19C 


60 


97 


85 


94.0 


„-9.0 



ERIC 
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Richfield Teacher Outcomes Study 
13-Year-Olds Package 1 (con.) 






Teacher Estimates 






Minimal 






Item 


Acceptable 


Desired 


Predicted 


— 

20A 


65 


97 


86 


20B 


72 


98 


91 


20C 


85 


99 


95 


21A 


71 


97 


85 


21B 


85 


99 


. 95 


21C 


85 


99 


95 


22A 


71 


97 


85 


22B. 


85 


99 


95 . 


22C 


85 


99 


95 


,23A 


71 


97 


85 


23B . 


85 


99 


95 


23C 


85 . 


99 


95 


2AA 


71 


.97 


85 


2AB 


85 


99 


95 


2AC 


85 


99 


, 95 1 - 



Actual 

Student 

Performance 




Teachers ' 
Prediction- 
Student 
Performance 



-3.9 
3.6 
7.6 

-3.2 

-0.6 
A. 3 
A. 2 

10.5 
1.3 
A. 2 
8.5 
7.2 
0.8 
9.0 

21.A 



ERIC 
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Richfield Teacher Outcomes Study 
13-Year-Olds Package 2 



v. 



ERLC 



r 1/ 

Item-' 



Minimal 
Acceptable 



Tea'ch er Estimates 

*r 



Desired 



Predicted 



Actual 

Student 

Performance 



1 


54 


91 


75 


2 


47 


' 85 


68 


3A 


80 


98 


93 


3B 


80 


98 

98 * 


93 


3C 


80, 


93 


3D 


80 


98 


93 


3E . 


£0 


98 


93 


3F 


/80. 


98 


93 


3G 


/ 80 


98 


93 


3H 


/ 80 


98 


93 


5A 


/ 70 
70 


95 


80 


5B 


95 


80 * 


5C 


70 " 


95 


80 


5D 


70 


95 


80 


5E 


70 


95 


80 


5F 


70 


95 . 


80 


5G 


70 


95 


80 


5H 


70 


95 


80 


6A 


84 


99 


92 


6B 


84 


99 


92 


6C 


84 


99 


92 


6D 


84 


99 


92 


6E 


84 


99 


92 


6F 


84 


99 


92 


6G 


84 


99 


92 


6H 


84 


99 


92 


7A 


55 


89 


77 


7B 


56 


. &8 


78 


7C 


62 


93 


81 


8A 


86 


99 


91 


8B 


86 


- 99 


91 


8C 


86 


99 


91 


8D 


86 


99 


91 


8E 


86 


99 


91 


8F 


86 


99 


91 


8G 


86 


99 


91 


811 


86 


99 


91 


9A 


81 


98 


90 


9B 


81 


98 


90 


9C 


81 


98 


90 


9D 


81 


98 


90 


9E 


81 


98 


90 


9F 


81 ' 


• 98 


90 


90 


81 


. 98 


90 


•911 


81 


98 


90 



Teacher 
Prediction- 
Student 
Performance 




82 
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Richfield Teacher Outcomes 


Study 






• 


13-Year-01ds 


Package 2 (con.) 


=~ ■ ■ ' 


r-T-, — r-. 










Teacher 






Teacher Estimates 


Actual 


Prediction- 


1 / 

Itenr-' 


Minimal 






. Student 


Student i 


Acceptable Desired 


rreaic tea 


Performance 


Performante 


11A 


91 


99 


yo 


92.6 


3.4 


11B 


83 


98 


on 
yu 


83j.9 


8.1 


11C 


76 


95 




4 3- 9 

90.2 


36.1 


11D 


85 


98 


Q1 

yi 


0.8 


12A 




90-, 


/O 


20.6 


57.4 


12B 


73 


94 


ftA 


71.8 


14.2 


12C 


81 


96 


Q1 

yi 


-74.0 


' 17.0 


13 


64 


93 


ftO 
OZ 


49.1 


32.9 


14 


72 


90 


7ft 
/O 


43.0 


35.0 


15 


56 


88 


/ J 


55.9 


17.1 


16 


58 


89 


77 


41.0 


36.0 


17A 


48 


75 


7 ^ 


58.0 


17.0 
0.9 


17B 


55 


84 


ox. 


80.1 


17C 


52 


80 


11 


65.2 


11.8 


17D 


51 


11 


1A 


73.2 


, 0.8 


17E 


54 


79 


7 7 


77.9 


-0.9 


17F 


54 


79 


/y 


85.6 


-6.6 


17G 


54 


. 81 


70 

/y 


83.8 


-4.8 


17H 


1 54 


81 


70 

/y 


74.4 


4.6 


18A 


; 71 


96 


ft7 

t 0/ 


71.4 


15.6 * 


18B 


5'9 


87 


7 ft 

/o 


33.1 


44.9 


19A 


78- 


98 


Q7 

0/ 


80.4 


6.6 - 


19B" 


78 


98 


o/ 


92.8 


-5.8 


19C 


7*8 


98 


* o/ 


95.2 


-8.2 


19D 


78 


98 


ft 7 


82.1 


4.9 


19E 


78 


98 


ft7 

0/ 


92.8 


, -5*8 


19F 


78 


98 


ft 7 
O / 


96.2 


-9.2 


19G 


78 


98 


" ft7 


71.5 


15.5 


1911 


78 


98 


87 


81.6 




2 OA 


31 


66 


49 


20.6 


28.4 


20B 


35 


61 


49 


52.3 


v\ -3.3 


20C 


46 


80 


69 


90.5 


-21.5 


20D 


45 


79 


73 


39.6 


33.4 


20 E 


57 


89 


74 


87.1 


-13.1 


20F 


61 


89 


76 


' 93.0 


-17.0 


20U 


63 


88 


79 


93.6 


-14.6 


20H 


51 


87 


77 


82.7 


-5.7 


21A 


78 


95 


88 


92.7 


-4.7 


21B 


79 


96 


92 


94.1 


-2.1 


21C 


77 


97 


92 


93.5 


-1.5 



ERIC 
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Richfield Teacher Outcomes Study 
13-Year-01ds Package 2 (con.) 



Teacher 



Itern^ 


Teacher 

Minimal 
Acceptable 


Eptimates 
Desired 


Predicted 

— % 


Actual 

Student 

Performance 


Prediction- 
Student 
Performance 


23A 


79 


98 


90 


87.3 


. 2.7 


23B 


' 85 


99 


94 . 




0.6 


23C 


84 


98 


93 


89.5 


3.5 


26A 


66 


91 


79 


42.5 


36.5 


26B 


73 


94 


85 


51.3 


33.7 


26C 


76 


9? 


93 


80.3 


. 12.7 



-Sterns 4A, 4B, 4C, 10A, 10B, 10C, 22A, 22B, 22C, 24A, 24B, 24C, 25A, 
25B, and 25C are identical to items 18A, 18B, 18C, 10A, 10B, 10C, 19A, 
19B, 19C, 23A, 23B, 23C, 7A, 7B, and 7C, respectively, in package 1. 
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Attachment Illb 

RICHFIELD TEACHER OUTCOMES STUDY 

Results by Domain, Objective, 
and Subobjective 
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Richfield Teacher Outcomes Study 
9-Year-Olds! 





NUuiDer 

of 
1 1 ems 


Richfield 
Student 
Performance 
% SE 


Predicted 
T.pvpI 


Predipted- 

Student 

Performance 


Minimal 

Acceptable 

Level 


- 

Desired 
Level 


Class 


TAT 


o 


92.2 


0.96 


97 6 


5.4 


94.4 


99.8 


N 


T A 0 

- lAz 


Q 


82-. 4 


1.28 




2. '6 


80.0 


90.0 


PN 


T A 1 


Q 

o 


92.7 


0.77 


76 T 


-1 6 4 


70.0 ^ 


81.3 


s 


T A C 


Q 
O 


82.6 


1.21 


Q 

do • y 


-15 7 


58 . 8 / 


78/8 


s 


T A /L 

lAo 


Q 


78.4 


1.45 




-22.8 


43.1 


68.8 


s 


IA* 


- — tr\ — 

40 


85.6 


0.61 


76 ^ 

/O.J 


^ • j 


69.3 


83.7 - 


s - 


IB1 


Q 
O 


73.6 


1.52 


Aft ft 




35.0 


56.3 


s 


IC1 


8 


67.0 


1.26 


66.3 


-0.7 


53.* 


73.1 


PS 


IC2 


8' 


47.8 


1.80 


43.1 


-4.7 


32.5 


56.3 




IC* 


16 


57.4 


1.10 


54.7 


-2.7 


43.1 


64.7 


PS 


I** 


64 


77.1 


0.59 


» 67.4 


-9.7 


: 58.4 


75.5 


s 












^ .... 








T T A 1 

I IA L 


o 
o 


57.6 


1.63 


60.6 / 


3.0 


48.1 


71.3 




T T A O 

IIA2 




81.8 


1.09 


79 6 

/ 7 . o 


-2.2 


73.8 


85.4 


PS 


IIA* 


20 


72.1 


1.16 


/ z . u 


—u • X 


DJ* J 


79 ft 




IIBl 


8 


70.5 


1.64 


A1 1 


-7 A 


Aft 1 


76 3 


PS* 


1IU1 


8 


82.2 


1.39 


78.1 


-4.1 


70.0 


88.8 


PS 


IID2 


8 


73^5 


1.43 


CO , ] 

58.1 | 


-17.4 


46. 3 


72.5 


s 


T ID* 


16 


' 78.8 


0.93 


68.1 


-10.7 


58.1 


80.6 


PS 




44 


74.3 


0.74 


69.0 


-5.3 


S3, 8 


79. 4 


PS 












? ' 
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Richfield Teacher Outcomes Study (continued) 











9-Year 


-Olds 


/ 


* 


DOS 


Number' 

of 
Items 


Richfield 
Performance 
% SE 


Predicted 
Level 


Predicted- 
Student 
Performance 


Minimal 

Acceptable 

Level 


De<5 ir ed 
Level " 


Class 


IIIA1 


21 


68.5 


0.92 


62.6 


-5.9 


51,0 


73.8 


PS 


IIIA2 


V 


54.3 


2.01 


60.0 


5.7 


47,5 


70.0 


PN 


IIIA3 


V 


62.5 


1.01 


55.9 


-6.6 


44.5 


68.6 


PS 


I) I A* 


34 


65.7 


0.85 


60.3 


-5.4 


48,7 


71.9 


PS 


IIIB1 


3 


67.1 


1.52 


s 56.7 


-10.4 


48,3 


68.3 


PS 


T T T no 


1 1 


69.8 


0.95 






55.0 


78.2 


PS 


XX XD 


-L H 


69^2 


0.93 


64. 6 


-4.6 


53.6 


76.1 


PS 


T T T * & 
111 ° 


^ Aft 


" 66.7 


0.81 


61.6 


-5.1 


50.1 


73.1 


PS 




















IVAl 


-ir 


68.7 


1.95 


77.5 


8.8 


70.0 


85.6 


FN 


IVA2 


si 


27.4 


1.70 


26.7 


-0.7 • 


16.7 


53.3 


PN 


IVA* 


11 


57.4 


1.60 


63.6 


6.2 


55.5 • 


76.8 


PN 


IVB1 


2 


73.7 


1.84 


72.5 


-1.2 


62.5 1 


82.5 


— 


IVC1 


8 


83.0 


1.44 


90.8 


7.8 


_J5£u3 ■ 


95.0 


N 


IVC2 


9 


63.3 


1.50 


65.0 


1.7 


51.7 


73.3 


— 


IVC3 


o 
J 


62.5 


1.60 


/O.J 


1 5 8 


71.7 


86.7 


N 


tve* 


20 ' 


71.0 


0.96 


77.3 


6.3 


68.5 


84.0 


PN ' 


IV** 


33 


66.7 


0.82 


81.5 


14.8 


63.8 


72.5 






















TOTAL 


189 


72.0 


0.58 


67.2 


-4.8 


57.3 


76.9 


PS 
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Richfield Teacher Outcomes Study 
13-Year-Olds 



DOS 


Number 

of 
I terns 


Richfield 
Student 
_Pc r for mane c 
% ~ SE^ 


Predicted 
Level 


Predicted- 
Student 
Perf ormance 


Minimal 

Acceptable 

Level 


Desired 
Level 


Class 


IA1 


8 


90.7 0.65 


95.0 


4.3 


93.0 


100.0 


N 




g 


88.1 0.73 


93.0 


4.9 


* ' 80.0 


98.0 




IA3 


'A 

8 


95.6 0.45 


91.0 


-4.6 


86.0 


99.0 


PS 


IA5 


8 


87.3 0.62 


75.1 


-12.2 


46.9 


95.0 


o 

PS 


IA6 


8 


86.6 0.67 


87.0 


0.4 


78.0 


98.0 


— mm 


1A* 


40 


88.9 0.33 


88.2 


-0.7. 


76.8 


98.0 


PS 


1111 


8 


87.4 0.58 


70.0 


-17.4 


50.0 

=■ — ; — -- 


90.0 


PS 


1C1 


8 


80.0 0.75 


80. -0 


0.0 


70.0 


95. U 




IC2 


8 


72.6 1.18 


74.0 


1.4 


50.0 


00 . U 


DC 


IC* 


16 


76.3 0.70 


77.0 


0.7 


60.0 


yu . _> 




j 


64 


86.0 0.34 


83.1 


-2.9 


69.2 


95. 1 


PS 






_r_- .j. '. — 










IIAl 


8 


72.3 O.90 


61.0 


-11.3 


46.0 


82.0 


PS 


IIA2 


8 


78.6 0.79 


81.5 


2.9 


61.9 


94.4 


• 


1 1 A* 


16 


75.5 0.76 


71 .3 


-4.2 
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